Nativescript-cli: Dev Dependencies (like Gulp, etc) getting built and build is failing because of which Gulp integration is not working currently.

[hdeshev]

_From @Headcult on April 26, 2016 11:30_

Thanks for the amazing work on integrating Angular 2 with NativeScript at such a good pace. Looking forward for future releases.

The main issue that I'm facing now is:
Dev Dependencies (like Gulp, etc) getting built and build is failing because of which Gulp integration is not working currently.

Can this be fixed at the earliest as we are unable to use Gulp within our project when using Angular 2 and NativeScript?

My Dev Dependencies:

"devDependencies": { "nativescript-dev-typescript": "^0.3.1", "shelljs": "^0.5.3", "typescript": "^1.8.9", "gulp": "^3.9.1", "gulp-debug": "^2.1.2", "gulp-typescript": "^2.12.1", "del": "^2.2.0", "run-sequence": "^1.1.4" }

Build Failure:

`:processDebugResources FAILED

FAILURE: Build failed with an exception.

• What went wrong:
  Execution failed for task ':processDebugResources'.

com.android.ide.common.process.ProcessException: org.gradle.process.internal.E
xecException: Process 'command 'C:\Android\android-sdk\build-tools\23.0.3\aapt.e
xe'' finished with non-zero exit value 1`

_Copied from original issue: NativeScript/nativescript-angular#195_

[hdeshev]

I finally reproduced the problem, and I believe this to be a problem with how nativescript-cli handles npm dependencies. Steps to reproduce:

1. Start an empty project.
2. Run it using $ tns run android
3. Install gulp: $ npm install gulp --save-dev
4. $ tns run android

Working around the problem is easy, but not obvious -- don't call npm, but let the tns tool do that for you by adding gulp to your package.json and running tns install or just tns run android

[PeterStaev]

Was struggling with the same problem! I can confirm that using tns install instead of npm install does the trick. Thanks @hdeshev !

[enchev]

Hey @rosen-vladimirov,

Can you post here more info why we have our own nmp and why we do not use the instilled one. What problems we solved?

Thanks

[rosen-vladimirov]

Hi @enchev ,
As you are asking, using npm as dependency resolves exactly this issue. In case we were relying on user's npm, you wouldn't have any workaround for this particular problem and in fact nativescript-cli would have been unusable with npm 3. And btw this is not "our" npm, it's the real npm, we are just using it as dependency.

How does npm work

NativeScript projects rely on npm and npm's structure. This means that each project has package.json with dependencies and devDependencies. When you call npm install all of your dependencies and devDependencies will be installed and placed in your node_modules directory. Different npm versions place the dependencies in a different way inside node_modules dir.
npm versions 2.x.x will install each dependency in it's own directory and each of it's dependencies will be installed inside it's node_modules directory. All dependencies of devDependencies will be installed in the respective node_modules dir.
npm versions 3.x.x tries to flatten the dependencies and install everything at your project's node_modules directory. So for example in case devDependency A has dependency B, which has dependency C, at the root node_modules you'll receive A, B and C directories.

How does NativeScript CLI handle dependencies

Due to known behavior of NativeScript's require, all dependencies used in a project must be flattened when copied to platforms/<platform>/.../tns_modules directory. So when the project is prepared, CLI tries to flatten the dependencies from node_modules dir, filters the one which are devDependencies or dependencies of devDependencies or dependency of dependency of dependency ... of devDependencies and process them to tns_modules dir inside platforms.

The problem

When npm 3.x.x is installed and user executes npm install, the flattening of modules makes it really hard for NativeScript CLI to understand from where is each package inside node_modules dir. It would require parsing all package.json files inside node_modules and it's subdirectories, checking each package and is it a dependency or devDependency, etc.

The workaround

NativeScript CLI has npm 2.x.x as dependency and executes npm install with it. This allows us to have the "old" way of installing the dependencies, where it is much easier to filter the devDependencies.

Why npm as dependency

In case we rely on user's npm version we have to:

• use child_process methods for spawning npm. Spawning new process is slower than simple require
• Parse stdout, stderr outputs of the newly spawned process. When we use the dependency, we work with callbacks directly where we have the result and error in a JSON objects.
• Your code should be usable with all possible versions of npm. However npm has breaking changes between different versions, for example some methods accept additional arguments in new versions. How would you handle this in your code?

[hdeshev]

Dragging our own pinned version of npm is a workaround that seems to work for many cases, but it's nothing more than a workaround. I believe we have to get rid of it -- the breakage caused by a command such as npm install (which is perfectly fine in any node project) is a real show stopper.

I'm probably oversimplifying everything, but can we rely on the user running npm install and use different module resolution strategies to identify modules we need to copy/update to <platform dir>/app/tns_modules?

[rosen-vladimirov]

Please excuse me, but relying on a specific version of a library is not a workaround for me. In the same manner we can drop all of our dependencies and ask the users to install them globally.

I agree that npm install is normal process and shouldn't break the project. We've already discussed this with @ErjanGavalji and he is working on a new approach that should resolve the issue with detecting which modules should be moved to tns_modules.

Also we cannot rely on user's calling of npm install - the idea was to have an easy way to do:

$ tns create app1
$ cd app1
$ tns run android

That's why CLI calls npm install on it's own. Also when preparing the project, we want to be sure everyhing is installed, so the user should not take care of this.
In case you decide that you want the users to call npm install, all of this will stop working.

[nraboy]

None of this seems to be working for me. I have two NPM libraries included in my package.json:

"bcryptjs": "^2.3.0",
"node-forge": "^0.6.39",

I install everything using tns install. I still end up with the following error:

* What went wrong:
Execution failed for task ':processF0F1DebugResources'.
> com.android.ide.common.process.ProcessException: org.gradle.process.internal.ExecException: Process 'command '/Users/nraboy/Library/Android-SDK/build-tools/23.0.3/aapt'' finished with non-zero exit value 1

Any ideas?

[lakshmankotipalli]

In my case, It resolved by removing the below line
<gap:plugin name="nl.x-services.plugins.socialsharing" />
from config.xml file

[enchev]

Hey @Plamen5kov

Do you know what's going on here?

[Plamen5kov]

Hi @nraboy
This is a problem with android tooling. The /Users/nraboy/Library/Android-SDK/build-tools/23.0.3/aapt is part of the android build-tools chain and it takes care of processing all project resources.
The problem is for the aapt tool, both files:

/node_modules/bcryptjs/dist/bcrypt.min.js.gz

and

/node_modules/bcryptjs/dist/bcrypt.min.js

are the same.
Your problem can be resolved if you delete /node_modules/bcryptjs/dist/bcrypt.min.js.gz file and rebuild the project.

[nraboy]

@Plamen5kov,

This method seems sketchy. Every time I add some NPM libraries I'm going to have to alter them from a package level? No one is going to want to do that.

Can't we get the NativeScript CLI to remove certain files? Seems like it wouldn't be too difficult to do.

Best,

[Plamen5kov]

Hi @nraboy,
Let me say again this is not a problem with NativeScript, but with the android toolchain. You are right, no one would like to remove random packages on a package level.
One possible fix would be to remove the problem file from the source package, so it can work with NativeScript without issue.
Another fix would be to get the NativeScript CLI to remove files, as you said, but I can't think of an easy way of guessing which files would cause problems, or any legit criteria by which to filter files. I you have any ideas, we are always open to suggestions.

[ErjanGavalji]

I'd vote against making the removal of specific files part of the CLI itself too. That would add an unnecessary dependency to the tool. A plugin might do the work instead, though I can't figure what its name would be - "ns-android-toolchain-fixer"? Sounds lame...

Can we figure out a better solution?

[PeterStaev]

I agree with @nraboy that there should be a process in the CLI to automatically clean up duplicate files. From what I read AAPT just removes the .gz extension of the file on add, so it should be relatively simple to traverse the modules dir and find duplicate .gz and non .gz files and just remove the .gz files.

[PeterStaev]

@ErjanGavalji , I see plugins as encapsulating functionality that is optional for the user. Here we are talking about a plugin which will be required if you want your build to run w/o problems. So I don't think a good thing to put inside a plugin. Also consider the situation of an existing project that was building fine, then you add a new module and it starts failing with some strange build errors. There will be no way for the end user to know that they need to install a plugin to fix those :)

[ErjanGavalji]

@PeterStaev You are absolutely right about the optionality of plugins and the hassle for plugin authors. The thing is, the problem happens only with android and only if you include some npm libraries. That's why I want to come up with something better.

So sad I can't figure out anything yet :/

[ErjanGavalji]

By the way, moved the dependent-npm related discussion to a separate thread.

[ErjanGavalji]

Well, @atanasovg, @Plamen5kov, how about having errors of the kind handled by the runtime via a CLI hook?

Can this be done as a general fix for any android tooling problems?

[Plamen5kov]

I think it's a great idea to build a tool that will make every node_module, NativeScript Compatible, but I don't think this tool should be CLI. As @ErjanGavalji said this problem concerns only specific node_modules, only in Android, and CLI should include only functionality concerning both platforms, when possible.
Furthermore if we only concentrate on deleting duplicate files, this solution will cover only one specific scenario and we should get a more general solution for the incompatible node modules.

I invite everyone to discuss all scenarios we can think of, concerning invalid node_modules, and find a more general solution, which we'll be able to support, keeping in mind the resources available to us.

[atanasovg]

If there is a valid way for us to uniquely detect such invalid cases for each Node module out there then we can think of some CLI hook, installed upon tns platform add android that will do some additional work prior to the tns prepare android command.

Out of curiosity - this is the first such case I am aware of, are there more such ones? If these are some finite number - e.g. 10 modules - then we may approach the problem in a different way.

[hdeshev]

Out of curiosity - this is the first such case I am aware of, are there more such ones?

The semver.js.gz issue (#1802) we're sometimes seeing for Angular projects is related even if it is caused by another bug -- semver is a dev dependency that should never end up in platforms/android anyway.

[Anilinfo2015]

Facing same problem , from last two weeks. no solution so far.

NativeScript log manager are very bad, it never tell you where the problems are? .
It only give higher level of errors/messages/warnings.

[Plamen5kov]

should be partially fixed with the merged PR