Nativescript-cli: Ability to run unit tests with `--bundle`

[rosen-vladimirov]

Is your feature request related to a problem? Please describe.
Currently I'm building and running my application with --bundle, i.e. with Webpack. However, this is not working for unit tests. This way I'm not sure my services are tested in the same way as when they'll be used at runtime in the actual application.

Describe the solution you'd like
Allow running tns test <platform> --bundle.

Describe alternatives you've considered
No

[Yermo]

I am also running into the same problem.

[cowboyd]

I still have the same problem using the latest version of the CLI. I created a repository to reproduce the issue https://github.com/cowboyd/nativescript-testing-spike/

[Fatme]

@cowboyd,

In order to test the feature you need the next versions of nativescript-dev-webpack and nativescript-unit-test-runner as well.

1. rm -rf webpack.config.js && npm i nativescript-dev-webpack@next --save-dev
2. npm i <path-to-nativescript-unit-test-runner.tgz> --save where path-to-nativescript-unit-test-runner.tgz package is downloaded from here

[cowboyd]

It looks like that link just takes me to the main repository for that packages. Are you meaning one of the tarballs like https://github.com/NativeScript/nativescript-unit-test-runner/tarball/master?

[Fatme]

Yes, that is what I mean - https://github.com/NativeScript/nativescript-unit-test-runner/tarball/master

[cowboyd]

Hmmm, I get the following error using both npm and yarn

$ yarn add -D file:///Users/cowboyd/Downloads/NativeScript-nativescript-unit-test-runner-v0.5.1-3-gd8cdbda.tgz
yarn add v1.13.0
[1/4] 🔍  Resolving packages...
[2/4] 🚚  Fetching packages...
[3/4] 🔗  Linking dependencies...
warning " > [email protected]" has unmet peer dependency "webpack@^2.0.0 || ^3.0.0".
warning "karma-webpack > [email protected]" has unmet peer dependency "webpack@^2.2.0 || ^3.0.0 || ^4.0.0-alpha || ^4.0.0-beta || ^4.0.0".
warning " > [email protected]" has unmet peer dependency "typescript@~3.1.1".
warning " > @angular/[email protected]" has unmet peer dependency "typescript@>=3.1.1 <3.3".
warning " > @ngtools/[email protected]" has unmet peer dependency "typescript@>=2.4.0 < 3.3".
warning " > @ngtools/[email protected]" has unmet peer dependency "webpack@^4.0.0".
warning "nativescript > nativescript-preview-sdk > [email protected]" has unmet peer dependency "typescript@>=2.1.0 || >=2.1.0-dev || >=2.2.0-dev || >=2.3.0-dev || >=2.4.0-dev".
warning "nativescript > nativescript-preview-sdk > tslint > [email protected]" has unmet peer dependency "typescript@>=2.1.0 || >=2.1.0-dev || >=2.2.0-dev || >=2.3.0-dev || >=2.4.0-dev || >=2.5.0-dev || >=2.6.0-dev || >=2.7.0-dev || >=2.8.0-dev || >=2.9.0-dev || >= 3.0.0-dev || >= 3.1.0-dev".
warning "nativescript-dev-webpack > [email protected]" has unmet peer dependency "typescript@*".
[4/4] 🔨  Building fresh packages...
[1/6] ⠄ fsevents
[2/6] ⠄ nativescript-angular
[3/6] ⠄ nativescript-dev-typescript
[4/6] ⠄ nativescript-unit-test-runner
error /Users/cowboyd/Code/NativeScript/testing-spike/node_modules/nativescript-unit-test-runner: Command failed.
Exit code: 1
Command: node postinstall.js
Arguments:
Directory: /Users/cowboyd/Code/NativeScript/testing-spike/node_modules/nativescript-unit-test-runner
Output:
internal/modules/cjs/loader.js:582
    throw err;
    ^

Error: Cannot find module '/Users/cowboyd/Code/NativeScript/testing-spike/node_modules/nativescript-unit-test-runner/postinstall.js'
    at Function.Module._resolveFilename (internal/modules/cjs/loader.js:580:15)
    at Function.Module._load (internal/modules/cjs/loader.js:506:25)

[cowboyd]

Dunno if it helps, but I attempted to clone the repository and link it for local development and got a different error:

ERROR in /Users/cowboyd/Code/NativeScript/nativescript-unit-test-runner/main-view-model.ts
Module build failed (from ../node_modules/@ngtools/webpack/src/index.js):
Error: /Users/cowboyd/Code/NativeScript/nativescript-unit-test-runner/main-view-model.ts is missing from the TypeScript compilation. Please make sure it is in your tsconfig via the 'files' or 'include' property.
    at AngularCompilerPlugin.getCompiledFile (/Users/cowboyd/Code/NativeScript/testing-spike/node_modules/@ngtools/webpack/src/angular_compiler_plugin.js:752:23)
    at plugin.done.then (/Users/cowboyd/Code/NativeScript/testing-spike/node_modules/@ngtools/webpack/src/loader.js:41:31)
    at process._tickCallback (internal/process/next_tick.js:68:7)
 @ /Users/cowboyd/Code/NativeScript/nativescript-unit-test-runner/main-page.js 2:15-43
 @ ../node_modules/nativescript-unit-test-runner sync (root|page)\.(xml|css|js|ts|scss)$
 @ ./main.ts

ERROR in ../node_modules/mkdirp/test/root.js
Module not found: Error: Can't resolve 'tap' in '/Users/cowboyd/Code/NativeScript/testing-spike/node_modules/mkdirp/test'
 @ ../node_modules/mkdirp/test/root.js 4:11-25
 @ ../node_modules/nativescript-unit-test-runner sync (root|page)\.(xml|css|js|ts|scss)$
 @ ./main.ts

[Fatme]

Hey @cowboyd,

Thank you for sending the demo repo. We really appreciate it.

I suspect the above errors are due to the fact that nativescript-dev-webpack is not working correctly with yarn. There are some known issues about that.

I successfully run tests using the steps below:

1. git clone [email protected]:cowboyd/nativescript-testing-spike.git
2. tns test init
3. Install unit-test-runner from https://github.com/NativeScript/nativescript-unit-test-runner/tarball/master
4. rm -rf node_modules
5. yarn install
6. yarn test ios

[cowboyd]

Install unit-test-runner from https://github.com/NativeScript/nativescript-unit-test-runner/tarball/master

I am still not able to successfully perform this step with either npm or yarn.

I can get it to install if I say, yarn add [PATH_TO_TGZ] --ignore-scripts

But then, it fails again with

WARNING in ../node_modules/nativescript-unit-test-runner/main-page.ts
Module build failed (from ../node_modules/@ngtools/webpack/src/index.js):
Error: /Users/cowboyd/Code/NativeScript/testing-spike/node_modules/nativescript-unit-test-runner/main-page.ts is missing from the TypeScript compilation. Please make sure it is in your tsconfig via the 'files' or 'include' property.
The missing file seems to be part of a third party library. TS files in published libraries are often a sign of a badly packaged library. Please open an issue in the library repository to alert its author and ask them to package the library using the Angular Package Format (https://goo.gl/jB3GVv).
    at AngularCompilerPlugin.getCompiledFile (/Users/cowboyd/Code/NativeScript/testing-spike/node_modules/@ngtools/webpack/src/angular_compiler_plugin.js:752:23)
    at plugin.done.then (/Users/cowboyd/Code/NativeScript/testing-spike/node_modules/@ngtools/webpack/src/loader.js:41:31)
    at process._tickCallback (internal/process/next_tick.js:68:7)
 @ ../node_modules/nativescript-unit-test-runner sync (root|page)\.(xml|css|js|ts|scss)$ ./main-page.ts
 @ ./main.ts

WARNING in ../node_modules/nativescript-unit-test-runner/test-run-page.ts
Module build failed (from ../node_modules/@ngtools/webpack/src/index.js):
Error: /Users/cowboyd/Code/NativeScript/testing-spike/node_modules/nativescript-unit-test-runner/test-run-page.ts is missing from the TypeScript compilation. Please make sure it is in your tsconfig via the 'files' or 'include' property.
The missing file seems to be part of a third party library. TS files in published libraries are often a sign of a badly packaged library. Please open an issue in the library repository to alert its author and ask them to package the library using the Angular Package Format (https://goo.gl/jB3GVv).
    at AngularCompilerPlugin.getCompiledFile (/Users/cowboyd/Code/NativeScript/testing-spike/node_modules/@ngtools/webpack/src/angular_compiler_plugin.js:752:23)
    at plugin.done.then (/Users/cowboyd/Code/NativeScript/testing-spike/node_modules/@ngtools/webpack/src/loader.js:41:31)
    at process._tickCallback (internal/process/next_tick.js:68:7)
 @ ../node_modules/nativescript-unit-test-runner sync (root|page)\.(xml|css|js|ts|scss)$ ./test-run-page.ts
 @ ./main.ts

ERROR in ./main.ts
Module not found: Error: Can't resolve '../node_modules/nativescript-unit-test-runner/bundle-app.js' in '/Users/cowboyd/Code/NativeScript/testing-spike/src'
 @ ./main.ts 9:8-78
Webpack compilation complete. Watching for file changes.
Webpack build done!
Hook skipped because either bundling or livesync is in progress.
Preparing project...
Project successfully prepared (iOS)
Unable to apply changes on device: 077A9294-6ED9-47A3-8C04-C8ED195E347C. Error is: Cannot find module 'nativescript-unit-test-runner/./lib/after-prepare.js'

[Fatme]

1. git clone [email protected]:NativeScript/nativescript-unit-test-runner.git
2. cd nativescript-unit-test-runner
3. npm i && npm pack -> this will produce .tgz in the current directory
4. Go to project dir and npm i <path-to-tgz-from-step-3> --save

[cowboyd]

That worked!

[cowboyd]

When is the plan to release these changes?

[Fatme]

Most probably the release will be next week.

[cowboyd]

So this worked in my testing spike, but when I tried to follow these steps in our actual application
it builds fine, but opens up and just hangs there.

You can see it on the cl/bigtesting branch of this repository https://github.com/resideo-platform/exemplar/tree/cl/bigtesting

[Fatme]

@cowboyd,

I've looked at the project and noticed that webpack.config.js file is not updated. You can update it using ./node_modules/.bin/update-ns-webpack --configs command.

PS:
You need always ensure that webpack.config.js is updated after changing the version of nativescript-dev-webpack plugin. This can be done by using one of the following approaches:

1. Execute ./node_modules/.bin/update-ns-webpack --configs after installing the new version of nativescript-dev-webpack plugin.
2. Execute rm -rf webpack.config.js before installing the new version of nativescript-dev-webpack plugin.

[cowboyd]

Ah, I see. Yes, I forgot that step.

One of the things that is difficult here is that we had to customize our webpack configuration to support accessing command line parameters from the built code.

As you can see, we tried to do this as unobtrusively as possible, by "decorating" the webpack config generated by nativescript-dev-webpack, but wouldn't it avoid situtations like this, if the boilerplate generated by nativescript-dev-webpack was extensible out of the box?

For example, it could look like this:

let base = require('nativescript-dev-webpack/webpack.config');
let extension = require('./webpack.enhance');
module.exports = env => {
  extension(base);
};

Then, there would be no need to run any update command to keep the boilerplate in sync, because it would be shipped with the npm module.

[Fatme]

Hey @cowboyd,

Thank you so much for your suggestion.

We really like the idea and will consider it for our next release.

[cowboyd]

@Fatme I'm still encountering this problem, even after following the steps to update the webpack.config.js You can see the newly generated one here https://github.com/resideo-platform/exemplar/blob/cl%2Fbigtesting/webpack.config.js

The app boots, but not into the test application. Instead it boots into the normal application, just as though I'd run yarn start ios

[cowboyd]

More information: for whatever reason, the env.unitTesting flag is not getting set by the CLI, so that the "unit testing" logic branch never gets executed.

If I manuall run yarn test ios --env.unitTesting=true, then it does bundle the test entry point, but it seems to hang connecting to karma.

Then, it eventually reports as connected, but doesn't seem to run the tests:

Here's the transcript from the console.

Successfully transferred bundle.js on device 3C910396-CEB5-4E0D-A0B3-7CB9A2341193.
Successfully transferred package.json on device 3C910396-CEB5-4E0D-A0B3-7CB9A2341193.
Successfully transferred runtime.js on device 3C910396-CEB5-4E0D-A0B3-7CB9A2341193.
Successfully transferred starter.js on device 3C910396-CEB5-4E0D-A0B3-7CB9A2341193.
Successfully transferred tns_modules on device 3C910396-CEB5-4E0D-A0B3-7CB9A2341193.
Successfully transferred tns-core-modules on device 3C910396-CEB5-4E0D-A0B3-7CB9A2341193.
Successfully transferred inspector_modules.js on device 3C910396-CEB5-4E0D-A0B3-7CB9A2341193.
Successfully transferred vendor.js on device 3C910396-CEB5-4E0D-A0B3-7CB9A2341193.
Restarting application on device 3C910396-CEB5-4E0D-A0B3-7CB9A2341193...
Successfully synced application com.resideo.exemplar on device 3C910396-CEB5-4E0D-A0B3-7CB9A2341193.
CONSOLE LOG file:///app/vendor.js:399:28: NSUTR: fetching http://127.0.0.1:9876/context.json
CONSOLE LOG file:///app/vendor.js:399:28: NSUTR: fetching http://192.168.86.63:9876/context.json
CONSOLE LOG file:///app/vendor.js:405:32: NSUTR: found karma at 192.168.86.63
CONSOLE LOG file:///app/vendor.js:228:20: NSUTR: connecting to karma at http://192.168.86.63:9876
CONSOLE LOG file:///app/vendor.js:405:32: NSUTR: found karma at 127.0.0.1
CONSOLE LOG file:///app/vendor.js:241:24: NSUTR: successfully connected to karma
25 03 2019 11:04:52.290:INFO [NativeScript / 12.1 (12.1; iPhone)]: Connected on socket EKzYZxbqrWmsw-yXAAAA with id NativeScriptUnit-7178
CONSOLE LOG file:///app/vendor.js:295:28: NSUTR-socket.io: ping timeout
25 03 2019 11:05:22.273:WARN [NativeScript / 12.1 (12.1; iPhone)]: Disconnected (0 times)Client disconnected from CONNECTED state (transport close)
NativeScript / 12.1 (12.1; iPhone) ERROR
  DisconnectedClient disconnected from CONNECTED state (transport close)

CONSOLE LOG file:///app/vendor.js:295:28: NSUTR-socket.io: 1
CONSOLE LOG file:///app/vendor.js:295:28: NSUTR-socket.io: 1
CONSOLE LOG file:///app/vendor.js:241:24: NSUTR: successfully connected to karma
25 03 2019 11:05:22.858:INFO [NativeScript / 12.1 (12.1; iPhone)]: Connected on socket ZzWrjilh5OdC8IvlAAAB with id NativeScriptUnit-3078

[Fatme]

@cowboyd,

More information: for whatever reason, the env.unitTesting flag is not getting set by the CLI...

Is this happen with the next version of NativeScript CLI?

[cowboyd]

It is using this pre-released version of the cli https://github.com/resideo-platform/exemplar/blob/cl%2Fbigtesting/yarn.lock#L4413

[Fatme]

@cowboyd,

This prerelease version of the cli doesn't support env.unitTesting flag. Here https://github.com/NativeScript/nativescript-cli/pull/4437/files#diff-3a043f40fea7608df4ef5e5d68237c27R111 is the PR that introduces env.unitTesting. It is merged 6 days ago so you need to update the version of the cli. I suggest you to update to the rc version of the cli - yarn add global nativescript@rc

[cowboyd]

Running yarn add -D nativescript@next worked!

As a side note, I avoid using global packages since I've got several different NativeScript projects kicking around and not having a global install means I can keep that variable constant.

[cowboyd]

Another thing that I noticed is that webpack is not picking up test files if they happen to be written in TypeScript. Is this a known issue?

[Fatme]

@cowboyd,

Yes, this is a known issue. You can find more info here https://github.com/NativeScript/nativescript-cli/issues/1798#issuecomment-477149608