Android-runtime: Initial builds of ng2 apps are slow

[tjvantoll]

Hey all,

While trying out the 2.0 release I noticed that the build time of my ng2 apps had gone up significantly—from ~1 minute to ~2–3 minutes.

More specifically, the build seems to hang at the Babel step:

:asbg:runAstParser
inputDir: /Users/tj/dev/nativescript/sample-Groceries/platforms/android/src/main/assets/app
outFile: ../bindings.txt
Building 40% > :asbg:runAstParser

It hung on that step for about a full minute, so I had actually assumed the build was stuck and was going to fail. As a small step I’d love to see some additional output while the Babel step runs—just so users don’t think something has went wrong—and as a bigger step it’d be great if we could find a way to speed up this build step, especially for ng2 apps.

Thanks!

[tjvantoll]

Also one other thing to note: I did confirm that subsequent builds are a lot faster, ~30 seconds for the app I was testing with. So this only seems to be a problem for the first build.

[Plamen5kov]

Hi @tjvantoll
The initial build is slow because we need to do a static analysis of all javascript files, which includes angular, which is quite big. The problem is we can't know at build time which javascript code is pure and which code has something to do with {N}, so we traverse every javascript file.
An easy workaround would be not to "analyze" angular at all, but that wouldn't be a general solution, because we would have to keep making exceptions for every pure js library.
A more general solution would be to find out which js code need to be "analized" and it's {N} specific and which is pure js and doesn't need to be "analyzed".

[blagoev]

A more general solution would be to find out which js code need to be "analized" and it's {N} specific and which is pure js and doesn't need to be "analyzed".

The only way to do this is to do the file parsing.

On dummy solution would seem to be to search for __extends to filter typescript code. For example there only 83 files in angular2 source dir out of 1091 js files there. This would not be working correctly since there may be interfaces extends in these files or the file may not be transpiled from Typescript.

The way to reliably know which files have defined native extends is to parse it and even this has drawbacks. (we are parsing Javascript after all)

[blagoev]

One possible way to handle this is to do the parsing at install time. * This will only mask the problem.

One other way is to provide plugin authors with packaging tooling to build and ship these static bindings as part of the plugin package. * This will rely on plugin authors to be "good citizens".

[Plamen5kov]

Another possible solution is to develop a convention where we say:
if you have pure javascript code put it in folder dontAnalyzeJsCode, and if you have ns related javascript code put it in folder analyzeJsCode.

[PanayotCankov]

I think most of the android specific code is in *.android.js files so you could use the suffix as convention. This will automatically filter pure JS libraries.

[blagoev]

@PanayotCankov this will artificially limit the developer to use *.android.js for android code. Having an if (android) statement in platform agnostic code might be desirable at times.

To my mind this is leaking the problem outside of the problem zone too much. The parsing is a performance feature to an extend, that all of a sudden requires changes to how I name my files. I am looking into this as a app developer that would rather have this sorted out by the framework. In that view for a plugin to be parsed at install time or at packaging/publishing time is perfect for me since I will not change it post install.

[petekanev]

@PanayotCankov In addition to what @blagoev said, say we were to follow a file naming convention, we could exclude all files but *.android.js, and that still leaves out the TypeScript files which will be compiled into *.js.

[PanayotCankov]

@blagoev We already had to recommend deleting the platforms file every once in a while due to bad caching that happens on other places.
@Pip3r4o *.android.ts files are compiled to *.android.js files, so I do not get your point here.

I do not insist on using the *.android.js it is just a good heuristic function.

P.S. The build time on my machine for blank angular app is 13 minutes so whatever you do to address this I'd be happy.

[petekanev]

@PanayotCankov I stand corrected, files indeed preserve their original name until copied to the assets/app folder, meaning we could traverse the app/ and node_modules instead, but the argument brought up by @blagoev still remains

[NathanaelA]

Since the NG2 is a plugin, maybe a json file could be included in the plugin that basically has all the info that you need. So we as plugin authors could generate that JSON file; and if it exists in that plugin folder then you use it; if not then you have to parse the plugin. By doing this with some of the larger plugins (like ng2) and tns-core-modules, you would eliminate the majority of the first time hit...

My thinking would be:
I as plugin author does a _tns plugin prepare_ then _npm publish_

• TNS code scans the JS files, and creates an entry in the package.json for the latest timestamp and generates the needed metadata file.

I as a consumer does a _npm plugin add somename_
Your code scans the js files for the latest timestamp; and checks the json entry:

• If invalid; it deletes the metadata.json and rebuild the metadata.json
• if valid; it leaves it alone.

By doing this check at plugin time; you first verify the data exists and is valid, number 2 the time is a much smaller hit while adding plugins, rather than the really long delay during the first build.

[PanayotCankov]

@NathanaelA Adding a key in the package.json to indicate your plugin has android code would be cool. And having prepublish script generated stubs for the extended classes will be awesome. That way the tns-core-modules can be shipped with no need of scanning and compilation at the client side, while pure JS npm packages will not be scanned since they do not have the extra key.

[petekanev]

What about a simpler solution - we read the package.json at the root level of each folder and if it doesn't contain a "nativescript": {...} key we skip traversal further down the tree and analyzing the files in the folder.

This means that we won't have to traverse any fat pure JavaScript frameworks or dependencies. In addition, if found necessary, there can be an extra custom key/flag in the package.json to instruct the binding generator against traversal of said package.

Also, in the near future, when generated files become part of the /app folder, we should be able to determine when and if we need to generate the classes again.

@PanayotCankov @NathanaelA @Plamen5kov @blagoev @slavchev @atanasovg

[blagoev]

@Pip3r4o :+1: perfect way to solving it.

It allows us in the future to choose parsing the runtime version in the package.json and switch between two different versions of the static binding generator if we need to.

[petekanev]

Good news, everyone! The fix is in the master branch and can be used by installing platform android@next. Please bear in mind that testing is under way, and it is possible that we have overlooked a very specific and unusual case.

[VladimirAmiorkov]

Awesome, I am seeing builds down to ~16 sec from ~200 sec on Android.