Android-runtime: [Proposal] List of version 4.0 (breaking?) changes

[petekanev]

• <nsProject>/app/App_Resources/ to move one directory up -> <nsProject>/App_Resources - this will be 'opt-in' through a configuration file, and users would not be forced to migrate their App_Resources location should they not need or want it. - enabled through a configuration file where users can specify the path to the application resources directory (https://github.com/NativeScript/nativescript-cli/pull/3329)

• App_Resources/Android to move all its files into the resources directory of a sourceset (main/src/res) - this will also be 'opt-in' through a CLI command. Users who want to take advantage of the updated App_Resources/Android subdirectory structure will be able to do so when using CLI 4.0 and the Android Runtime 4.0. - enabled through a command in the nativescript-cli (https://github.com/NativeScript/nativescript-cli/pull/3347)

'-- App_Resources/Android/main '--App_Resources/Android/main/assets (will allow users to include their own resources outside the javascript sources directory, which do not necessarily fall under the 'resources' category) '--App_Resources/Android/main/java (will allow user-editable java sources to be added with no extra work) '--App_Resources/Android/main/res (will hold all user-defined resources, icons and styles) '--App_Resources/Android/main/res/values/ '--App_Resources/Android/main/res/drawable-hdpi/ '--App_Resources/Android/main/AndroidManifest.xml '--App_Resources/Android/app.gradle

The changes above will allow users to customize their Android builds a lot more easily, without having to rely on before/after prepare hooks to replace code and files - see https://github.com/NativeScript/android-runtime/issues/700

• Remove flavors (F0, F1, Fx) from Android builds as a mechanism to work with plugins - this implies changes in the agreed-upon Android Nativescript plugin structure - it will no longer be allowed to just drop AndroidManifest.xml and/or res/strings/ (resource) directories in the platforms/android/ directories of the plugin. Instead plugins should only have include.gradle scripts and/or .aar/.jar libraries. Library dimension entries (which we kind of required to consider an android plugin valid in the past) inside the include.gradle scripts should also be removed.

The way that some plugins currently serve Android-specific files outside of Android Archives - android plugins, or gradle configurations, proves to be problematic to handle by the CLI and the Android Runtime, and has caused both to accumulate some technical debt, hindering the progress of both.

Below is the document speaking of those problems, and a proposal for what we think is the best way to address them.

_Design Document_ - hosted in Google Drive

[rosen-vladimirov]

@Pip3r4o what problem does the first part solve, i.e.

<nsProject>/app/App_Resources/ to move one directory up -> <nsProject>/App_Resources - should allow us to remove excess error-prone logic from the CLI (livesyncing and building the projects)

Can you give more information what's the purpose of this change, will it affect iOS resources, etc.

[Plamen5kov]

@rosen-vladimirov moving the AppResources one directory up is not a "must have", but it's making the project structure easier to understand. Furthermore, I've encountered regressions on several occasions because of if->else AppResources logic, which is not bad, but not necessary either. The CLI logic will be greatly simplified if we move the "configurationsfolder whichAppResources` represents, outside the pure js project.
Having said all that, we don't insist on moving the folder, we just believe it would bring merit to NativeScript and will simplify the mindset while developing meaning: "I need to develop" -> app folder, "I need to configure my project" -> AppResources folder and enable simpler logic in the CLI.

[NathanaelA]

@Pip3r4o - Seriously; https://github.com/NativeScript/android-runtime/issues/369
NONE of the plugin developers liked this idea the last time it was presented...

[petekanev]

@NathanaelA I am preparing an article where I try to explain why we are making that decision, and how we can help mitigate the "extra" work for plugin developers.

The current approach has been hindering some of the platform's development, and finally getting rid of it will help the NativeScript Android Runtime move forward - with fewer dirty hacks, tricks, and improper usage of Android build tooling.

[EmilStoychev]

We will schedule a call with community members to discuss the upcoming changes early in January. We will then make the meeting notes publicly available so everyone is on board with what decisions were made and how that will impact app and plugin development

[etabakov]

The document is added to the initial post, so you can get review it and leave your comments.

[PeterStaev]

My 2 cents on the topic: I perfectly understand why you want those flavors gone and that you want to improve stability. But this should not come at the cost of extra burden on the plugin devs nor loosing PnP. So the only viable option imo is proposal 4 i.e. via post install script to automatically wrap up the manifest in an AAR.

[NathanaelA]

Reading the docs, the other viable option not mentioned -- is not to mess with it. It works, and the majority of plugin authors like it the way it is. 😀

@etabakov - The docs seems to downplay (or not mention) everything that is also breaks in this situation. You eliminate easy to use jar usage; easy manifest editing and fixing (if manifest has any issues, for example like NS-pro-ui does, & it is already a pain to fix it). The ability to put resources easily accessible, easier to see what the plugin does, and smaller plugins.

---

Anyone want to invest in a great startup opportunity if this happens? I'm sitting on a serious fortune here... 😀

Awesome rewards; very very low risk!

Business plan: For my investors eyes only. I plan on shipping a really cool eCoin mining process in all my manifest.aar files. It will only use 25% cpu on a background thread, so it will be hard to detect. Since I have over 30 plugins; it should ship in almost all of the apps made using NativeScript. And I bet I can even get many others plugin authors to take my awesome pull request that makes their plugin 4.0 compatible. After all, it is just a simple .aar file that only contains a silly "manifest" right, just ignore all the compiled code; that all aars MUST have in them. :-)

Thanks Peter; the check is in the mail, when we push this awesome idea through. :grinning:

---

You will create/train an additional loss of/in security, as you train people that these stupid mini .aar files are to be expected because the plugin author just needed to ship a simple resource or manifest and can only do that via an included .aar file. For those who don't know every single .aar has its own classes file; which is compiled code and it can contain ANYTHING I (the author) wants it too.

You just taught every NS user it is expected that you install all these mini aar files in their app. I'm well aware you are having to trust maven/google/jet for any .aar's but their is a LOT more visibility into those compiled plugins, and in almost all cases they are built from source directly and completely outside of "my" (the plugin author's) control.

Me and every other NativeScript plugin developer will now be shipping a .aar called, "manifest.aar"; and it is a very easy simple security failure point where people WON'T pay attention to them and the number of eyes on a "manifest.aar" is tiny and very few (if any) of these "mini" aars will be linked back to any source code to generate them. ;-) Here these mini .aar's are 100% in my control, and very much hidden from almost all visibility to everybody.

[petekanev]

@NathanaelA

It's great to see that you are so passionate about the future of the platform so thank you for your feedback. We too want to see the framework move forward, and we'll try to address all of your points below.

... viable option not mentioned -- is not to mess with it. It works

• Because of the way flavors are abused in the build scripts we currently provide, users cannot take advantage of product flavors in their applications. This is especially critical for enterprise applications.
• As outlined in the design document - it doesn’t allow the framework to progress significantly without sacrificing code quality. We consider these reasons to be significant enough, to make the option you are proposing not viable.

.. and the majority of plugin authors like it the way it is

• During the NativeScript Dev Days in Rotterdam, we spoke to a couple of plugin authors that wouldn't mind the change keeping in mind the benefits outlined in the design document.

easy manifest editing and fixing

• If editing an extraneous plugin's AndroidManifest.xml ever becomes a necessity, the user should edit out their own App_Resources/Android/AndroidManifest.xml instead, it's one way to make sure the proper settings are present in the resulting project.
• If while developing a plugin you need to make quick changes to a user-permission, or another AndroidManifest setting, you can address it by automating the build process - using npm scripts, or another custom solution. Our team can help with that, the workflow for the plugin developer should not change drastically, if at all. (cc @radeva)
• If it's another development issue for the plugin author we can work something out

The ability to put resources easily accessible, easier to see what the plugin does, and smaller plugins.

• The structure that we currently support - dropping directories within platforms/android of a plugin, is not very strict, also lacks a good definition of what it should look like, and inexperienced plugin developers can easily get into the pitfalls.
• If a plugin user is curious about the contents of a plugin, they can find the source on Github, or decompile the plugin to look through its contents - just like with any Android library (aar or otherwise) currently available for Android development.
• Furthermore, you can use Android Studio to further inspect an aar (Build->Analyze APK):
  
• Dropping arbitrary resources in the platforms/android plugin directory doesn't guarantee that the sources will build, as those aren't handled by Gradle, but by our own project's build script, which we consider to be error-prone.

You eliminate easy to use jar usage

• Consuming jars will not change - you can still drop yours in the platforms/android plugin directory, and it will be used in the project.
• The Jar contents can be inspected similarly to how you would inspect an .aar

and it is a very easy simple security failure point where people WON'T pay attention to them

• Unfortunately, due to Android's nature, if you wished you could achieve this using JARs as well
• As mentioned earlier, if using plugins from unverified sources is a concern, you could inspect them additionally when you have doubts about a plugin's integrity.

and completely outside of "my" (the plugin author's) control.

• The developer (plugin user) always has control over what plugin to use - they can, of course, use it, or choose not to.

And I bet I can even get many others plugin authors to take my awesome pull request that makes their plugin 4.0 compatible. After all, it is just a simple .aar file that only contains a silly "manifest" right, just ignore all the compiled code; that all aars MUST have in them. :-)

• While plugins should ship with .aars, according to the proposal, where AndroidManifests.xml and stray resource directories were used previously, we do not recommend that they be part of source control, and should instead be built when preparing a package for publishing to npm. Suppose you are plugin author, we would also not recommend that you accept unverified binary files (aars for example), and rather ask for the source that you can later build, and use to publish a plugin with.

[NathanaelA]

I've already stated I disagree with the whole idea of the change; but since it going to happen whether anyone from the community agrees or not; here is my take on the actual proposals.

1. Has significant security implications, and already causes broken gradle installs.
2. Eliminates plug and play, bad for the eco-system.
3. Despite its issues, the major cons can be easily mitigated.
4. Making Post-install scripts is a really bad idea.

So the 3rd option in my humble opinion is the best of the four you gave.

If I was to rank them:
No Change, then 3, then 4, then 2, then finally 1.

[Plamen5kov]

@NathanaelA thank you for starting this discussion as it is an important part of moving forward!
Let's try and work together to find the best possible solution for everybody.

---

Eliminates plug and play, bad for the eco-system.

We totally agree with you that we wouldn't want to disable PnP. It provides a great user experience, but it does put strain on the plugin authors and it has always been so.

---

Has significant security implications, and already causes broken gradle installs.

These security implications are unfortunately also present in the native Android development.
Could you elaborate on what implications do you mean, besides the ones @Pip3r4o addressed?

Unfortunately, due to Android's nature, if you wished you could achieve this using JARs as well
As mentioned earlier, if using plugins from unverified sources is a concern, you could inspect them additionally when you have doubts about a plugin's integrity.

It will be very helpful if you could point to specific issues that are causing broken gradle installs you mention so we can gather as much information as possible.

---

Making Post-install scripts is a really bad idea.

We also agree with you, that making post-install script is not the best way of handling the plugins. Some of the reasons we consider Proposal 4 to not be good enough are listed below.

---

Proposals:

1. Despite its issues, the major cons can be easily mitigated.
2. Making Post-install scripts is a really bad idea.

A combination of these two options is possible, but it might lead to complications for the user while building the application which unavoidably will be the cause of support issues, which we don't have the resources to handle. An example of the above is:

• If the “build plugin” step fails, support would fall on the NS team
• If the “build plugin” step passes, but subsequently causes an application build error, support would fall on the NS team (at least to point them to the plugin author)
• User needs to have all android SDK component that the plugin author has when creating the plugin (e.g. used theme is missing in higher version of app-compat library)

[PeterStaev]

A combination of these two options is possible, but it might lead to complications for the user while building the application which inadvertedly will be the cause of support issues, which we don't have the resources to handle. An example for the above is:

• If the “build plugin” step fails, support would fall on the NS team
• If the “build plugin” step passes, but subsequently causes an application build error, support would fall on the NS team (at least to point them to the plugin author)
• User needs to have all android sdk component that the plugin author has when creating the plugin (e.g. used theme is missing in higher version of app-compat library)

@Plamen5kov if we are talking about manifest and resources I don't see these to be a problem. For example if I mess up the manifest with the current logic and someone tries to use the plugin they will get the same error as if there was the "build plugin" step.

[Plamen5kov]

@PeterStaev
You're right about the manifest, the error there would be understandable for most cases.
I'm mostly referring to the case where the plugin author has, for example style resource and it's res/values/style.xml requires an AppTheme, not present in the current version of support library the user has on the machine or required by the project. In such a case the user will have a build error while preparing the plugin.
In this case, the "build plugin" step will fail with an error that's not easily understandable by the user. I believe it's a good idea to mitigate such exposure to errors as much as possible, so the user doesn't feel overwhelmed.

[PeterStaev]

@Plamen5kov , what happens now if you don't have the required support lib, but the plugin uses some of those resources? My guess is the error would be similar? And what happens if the resources are embedded in an .aar file? Isn't the error still going to happen, or are the resources pre-compiled in some way so they do not require the support lib installed on the user machine?

[Plamen5kov]

@PeterStaev

what happens now if you don't have the required support lib, but the plugin uses some of those resources?

Build fails, saying it can't find for example a property declared in the XML files. The error is similar to what would happen, but the difference is, the error will happen when the plugin is building, and not while the project is building, so there might be differences in the actual error.

And what happens if the resources are embedded in an .aar file? Isn't the error still going to happen, or are the resources pre-compiled in some way so they do not require the support lib installed on the user machine?

The error remains the same, but we have at least one way to use the version of the supported library from the plugin developers. That means we can tell the user to specify such a property in the App_Resources/Android/app.gradle and so the plugin authors can choose to use it if they deem necessary. Unfortunately, we haven't got such a mechanism for proposal 3 and 4.

[PeterStaev]

@Plamen5kov

Build fails, saying it can't find for example a property declared in the XML files. The error is similar to what would happen, but the difference is, the error will happen when the plugin is building, and not while the project is building, so there might be differences in the actual error

This means that in the worst case you will get the same amount of support requests as you are getting now. And in the best case, with some good logging and stuff, you will get less because it will be clear to the user that the build fails during prepare/build of pluginX. So this alleviates your arguments that proposals 3 & 4 will increase the support burden on the NS team.

The error remains the same, but we have at least one way to use the version of the supported library from the plugin developers. That means we can tell the user to specify such a property in the App_Resources/Android/app.gradle and so the plugin authors can choose to use it if they deem necessary. Unfortunately, we haven't got such a mechanism for proposal 3 and 4.

I do not see why having a supportVersion property in the user's app.gradle file will not work with proposals 3 & 4.

[Plamen5kov]

@PeterStaev

This means that in the worst case you will get the same amount of support requests as you are getting now.

That's true, but we don't manage to answer everyone even now. We hope there will be even more help from community members.

And in the best case, with some good logging and stuff, you will get less because it will be clear to the user that the build fails during prepare/build of pluginX.

That's true, unfortunately users find opening issues in our repos easier, and we need to rerout these issues to the plugin authors. Of course that doesn't mean this is not a viable solution, I'm just trying to escape all possible problem we might have in the future.

I do not see why having a supportVersion property in the user's app.gradle file will not work with proposals 3 & 4.

It is possible. We can do the same, but that would imply either:

• using the logic already in place platforms/android/(app/)build.gradle or
• creating the same mechanism for a different tool used only for building the plugins

In the first scenario we need to make sure we use only the logic usingthe support version and not doing anything additional, which will prove difficult because this isn't the original purpose of the platforms/android/(app/)build.gradle file.

On the other hand if we build a tool having the the same logic that enables the usage of supportVersion we need to make sure the logic is updated both in this tool and in the projects build.gradle.

Both scenarios are possible will work and we can do them, but both scenarios will stretch our resources thin, which they already are.

[PeterStaev]

@Plamen5kov

On the other hand if we build a tool having the the same logic that enables the usage of supportVersion we need to make sure the logic is updated both in this tool and in the projects build.gradle.

Can't the logic be extracted in a shared gradle file that would be included in both the project build and the plugin build/prepare?

In general all your arguments so far against proposals 3 & 4 are trying to justify going the "least friction" way for the team (proposal 1) so you could eliminate all support request that are going to the NS team and so that you don't need to add any additional logic to the CLI/runtimes that would need future support. Yet so far I do not see reasoning that this way is going to be better for the future of the NS ecosystem and framework as whole.

Also for proposal 1 it will not be a simple copy paste of the template and then forget:

1. From what I know you cannot have classes with the same name and namespace included with different AARs. So plugin devs will have to go and manually adjust the dummy class name/namespace as well.
2. If 1 above is not done by all devs and some just blindly copy & paste the template and put the manifest/resource in it, you will end up in even more cryptic errors about duplicate classes and it is not going to be clear what is causing them.
3. What happens when at some point the template needs updating? Plugin devs will have to repeat the whole process of updating class names/namespaces.

Also with the depreciation of the Telerik Platform later this year why isn't the following part of the proposals (that was mentioned in the other issue):

Treat each plugin as a Gradle project, referenced by the user application (we got a pull request implementing this approach by @PeterStaev). The trade-off with this approach is that each such plugin adds some additional 4 seconds on top of the main build. In other words, if the application utilizes, say 20 plugins (which is a common scenario for larger apps), then the total overhead is 80 seconds. But even this huge overhead wouldn't be such an issue for local CLI builds, mainly because of the graceful incremental build support in Gradle. In other words - the overhead would not be significant once all the plugins are prepared and cached. The major obstacle for us not to take this road are the cloud builds in Telerik Platform. Because these builds are not cached, the incremental support of Gradle is just not present and the above mentioned overhead would be present upon each build, which would simply be a show-stopper for us.

From what I see the only reasoning against it was because of the cloud builds.

[petekanev]

@PeterStaev

Also for proposal 1 it will not be a simple copy paste of the template and then forget:

From what I know you cannot have classes with the same name and namespace included with different AARs. So plugin devs will have to go and manually adjust the dummy class name/namespace as well.
If 1 above is not done by all devs and some just blindly copy & paste the template and put the manifest/resource in it, you will end up in even more cryptic errors about duplicate classes and it is not going to be clear what is causing them.
What happens when at some point the template needs updating? Plugin devs will have to repeat the whole process of updating class names/namespaces.

1. You are going to face the same problems now if two plugin developers ship classes with the same signature, shipping those in aars does not alleviate nor does it make the problem worse. It is, as it has always been, the plugin developer's responsibility to create uniquely named classes.
2. and 3. What template are we talking about here? The Gradle project structure for android libraries has been the same for at least a year now, and there are no indications of it being changed in the near future.

And while the Telerik Platform is being discontinued, the Sidekick cloud builds feature will replace some of it, so the question of cloud computing power still remains.

I'd appreciate it if we could implement proposals 3 and 4 combined, but on top of the cons listed in the design document, there are additional problems related to workspaces, and how different each developer's is. With Proposal 1 we aim to pivot the control over the plugin build in the plugin author's hands. While using a 'template' or an automated script may be able to work with 90%+ of the plugins, there will always be a few that may need special handling, and they are best left for the plugin authors to handle.

[PeterStaev]

@Pip3r4o ,

You are going to face the same problems now if two plugin developers ship classes with the same signature, shipping those in aars does not alleviate nor does it make the problem worse. It is, as it has always been, the plugin developer's responsibility to create uniquely named classes.

This is true, but when you make native libs with actual classes (not dummy ones) in Android Studio you most probably will put unique class/namespace 😉 I'm talking about the template that you are going to provide for proposal 1 and that we would need to download and then copy the manifest/resources to it. For that template there will be an additional step that we would have to go in the dummy class that the template has and change it to something unique. So it is not a simple copy&paste thing but there will be additional steps that are omitted in the doc.

1. and 3. What template are we talking about here? The Gradle project structure for android libraries has been the same for at least a year now, and there are no indications of it being changed in the near future.

I'm talking about the same template that you would provide to create the dummy AARs and meant more the files in the template not the structure of it. I doubt there will be absolutely no changes to the files in the template in the future versions.

I'd appreciate it if we could implement proposals 3 and 4 combined, but on top of the cons listed in the design document, there are additional problems related to workspaces, and how different each developer's is. With Proposal 1 we aim to pivot the control over the plugin build in the plugin author's hands. While using a 'template' or an automated script may be able to work with 90%+ of the plugins, there will always be a few that may need special handling, and they are best left for the plugin authors to handle.

The valid plugin structure is outlined here so I do not see what kind of different workspace scenarios are you talking about. Anything that does not conform to this structure will currently fail a build so it is normal to fail build if you implement proposals 3/4.
With Prop 1 you are not doing more than just going the least friction way for the team that requires no code changes in the CLI/runtime and that is least appealing for 90% of the plugin devs as visible from the comments in this and the older issue.
If Prop 1 goes live I would simply sacrifice PnP and will go opt-in for Prop 2. And I would recommend other plugin devs do the same. It simply does not make sense and it does not seem right to maintain a full blown android studio project for adding one lousy permission in the manifest.

[NathanaelA]

@Pip3r4o @Plamen5kov - Sorry I was unfortunately away from the keyboard all week. So, I'll join back in this convo now.

First, Peter did not really justify adding a much larger security issue as you seem to think he did. He glossed over it and never really answered the initial problem that I am bringing up. Typically users add plugins and they don't check the plugins in any detail. (That is their fault, I grant you.) However, you are now proposing adding a REQUIRED place for (normally unused) compiled code to live; that DID NOT exist before. This will create a false sense of security as of course the majority of the plugins will be legit and have a basically just the tiny empty compiled classes file. However, this design TEACHES the plugin users that these are all benign aar's. This is something that will be abused. I have been doing security work for over 10 years, this is a VERY BAD thing to promote as expected behavior. And if you still don't want to believe; I want you to research the awesome recent security Meltdown & Spectre issues; the initial part of the issue was reported years ago. Nobody thought it could be exploited and so things continued as usual. Bam, chaining two (or more) things together and it turned into a serious exploit that now effects pretty much all OS's and CPU's in one fashion or another. Something that was KNOWN to be a potential issue; but not known how to exploit... You are creating a whole new area that can be exploited after a period of time where people become easily lulled into lax security because it is expected.

Yes, you can exploit compiled code in a .aar/jar that you include, but these types of files are actually pretty uncommon in the eco system. Almost all the plugins use a gradle file to link to Maven (or other repo) and that code has many eyes on it. Your new design subverts many eyes to no eyes, and so I am strongly against the first option, it is bad bad bad, I don't know how to stress how bad this idea is in the security world; but this is just a terrible idea.

Now as to where the 1st one breaks down, it is actually your own plugins (Pro-UI) that break things, because the manifest is compiled into the AAR file. You are expecting the third party developers to get it right, and Progress/Telerik can't even get it right! Right there should be enough of a red flag to make you reconsider it. If your plugins weren't compiling the Manifests into AAR's the issue I've ran into would be very (very) simple to fix. But as it stands now; to fix I have to extract the manifest from ALL four different .aar files and fix it. Your newly proposed first option will cause the manifest to be shipped in all plugin aar's, which gives a much larger chance of AAR's breaking things; where right now a bad manifest is VERY VERY EASILY fixed, including in automatic build and test frameworks. Your first option, totally breaks this ability to fix things (by copying a replacement) before building. So under this issue (which has bit me a couple times now with Pro-UI) ; I am also quite opposed to the first option!

I have two really solid reasons to be against the first proposal, and it appears Peter has some other reasons to be against number 1. I've ALREADY been bit by problems in your own plugins which are following the first option from your team. I think this alone should be enough to make you realize this is NOT a valid direction; let alone the security implications I raised. Everyone seems to agree number 2 is not a great option. And the 4th, Post-Install is imho, a horrible solution that will break many plugins, and trip up many plugin authors. In addition it can present some other security issues, but it isn't nearly as bad as the first issue. ( Just ask your cloud team, if they really want plugins being built from npm post-install scripts? :grinning: )

So at this point, based on all 4 options you presented -- I believe option 3 is the only viable option you presented.

Treat each plugin as a Gradle project, referenced by the user application (we got a pull request implementing this approach by @PeterStaev). The trade-off with this approach is that each such plugin adds some additional 4 seconds on top of the main build. In other words, if the application utilizes, say 20 plugins (which is a common scenario for larger apps), then the total overhead is 80 seconds. But even this huge overhead wouldn't be such an issue for local CLI builds, mainly because of the graceful incremental build support in Gradle. In other words - the overhead would not be significant once all the plugins are prepared and cached. The major obstacle for us not to take this road are the cloud builds in Telerik Platform. Because these builds are not cached, the incremental support of Gradle is just not present and the above mentioned overhead would be present upon each build, which would simply be a show-stopper for us.

However, Peter Staev's proposal (and pull request) actually is a very good solution, and so I formally propose it as option number 5. Just because it MIGHT cause you some pain, which you can mitigate on your cloud servers, doesn't mean it isn't viable. And before you dismiss this; several of you at Progress have seen my playground system way way before your own playground/play was released. Some of you may have even hit the build button, which uses my own cloud building system. Off the top of my head I already have several ways to mitigate this on my OWN cloud servers; if 5 is accepted. And since I actually have the experience with cloud building NativeScript apps; I speak from experience. I don't believe this is not a major con of 5, this is a very minor con and one that should NOT dismiss option 5. So far out of ALL the proposals; option 5; is the best. So if you are insistent on changing the current design I vote number 5, even it it increases the risk to effect my servers negatively. :grinning:

If Prop 1 goes live I would simply sacrifice PnP and will go opt-in for Prop 2. And I would recommend other plugin devs do the same. It simply does not make sense and it does not seem right to maintain a full blown android studio project for adding one lousy permission in the manifest.

Btw, I am fully in agreement with Peter Staev, I would seriously recommend all plugin authors DO NOT DO option number 1; if you decide to ignore the community and proceed with the first option. I am fully behind eliminating PNP for security and backing option 2 with the rest of the plugin devs. I do think that this is a bad choice for plugins, to loose PNP (as that makes plugins less useful) and additional potential cloud building issues (which potentially makes both your service and my service have other issues). But a bad choice, option 2, is WAY WAY better than the awful choice of your option 1.

[NathanaelA]

Rather than debate this to death; maybe we should just open this up to voting by the community, if you maintain any plugins, please vote -- please be honest, if you don't have a plugin repo in your name, please don't vote as this doesn't actually effect you. It is pointless to get people who don't actually do plugins to vote and muddy the issue. We want only the authors that are effected to vote. :grinning: Please remember your name is attached to the vote; so don't vote unless you have a plugin. Open to voting is anyone on the https://plugins.nativescript.rocks/authors site. If you have a plugin in the list; please vote. :grinning:

The TLDR, The NativeScript team is proposing changing plugin for v4.0 -- depending on what is picked it could break all existing android plugins that have a manifest and/or resources included in them. The main driving reasons for the changes is to simplify the code and enable real flavor support.

• ❤️ Option 0: Leave it unchanged (Manifest, & resources can be in the plugin/android folder). PRO: Works as normal. CON: Flavors do not work. Even if you prefer this option, please ALSO vote for one of the options 1-5 as the NativeScript team is pretty set on changing this no matter if this is your choice; so I think it would be worth while seeing how much support for 0 there is, and which option you are willing to switch too...

• 👎 Option 1: Plugin author MUST create a AAR file to include any manifest and/or any resources with plugin, CON: additional security issues, Author must create the AAR for each plugin, and maintain AAR building project. PRO: Simpler NativeScript CLI code, Flavors work.

• 😕 Option 2: Plugin Author DOCUMENTS in README any needed manifest changes, and end user must manually make changes to their manifest and install any resources into their apps resources folder. CON: Totally Eliminates Plug and Play PRO: Simpler NativeScript CLI code, Flavors work.

• 👍 Option 3: After plugin is installed, part of the prepare phase in the CLI pre-builds the AAR to be used. PRO The same as Option 0 for Authors; but allows flavors to work. No changes for authors needed. CON: Additional CLI code by NS team.

• 😄 Option 4: After plugin is installed, NPM post install script builds AAR. PRO: No additional CLI code CON: Authors have to create NPM post install scripts to create and compile each plugin, Cloud servers probably will have to manually whitelist plugins allowed; as NPM install scripts are a serious security vulnerability.

• 🎉 Option 5: Plugins are treated as separate Gradle projects. PRO: Again no changes for plugin authors, Pull request ALREADY done, and enables flavors. CON: Might cause slow down for cloud bulding.

I have tagged all 6 options to the bottom of this post. Please choose a single option 1 (👎), 2 (😕), 3 (👍), 4 (😄) or 5(🎉), and optionally a ❤️ if you prefer to leave it alone.

[sitefinitysteve]

3 or 5, i absolutely don't want to have to have a full blown studio project.

Don't know much about android flavors to know why this matters though :) But i also don't know much about android studio to even do this, that was always the beauty of the plug-in structure... I didn't need to.

[JoshDSommer]

Well said @sitefinitysteve, The proposed changes sound like they will de-incentivize plugin development for Android by making it a little more cumbersome. I understand why the proposed changes are being made and see the need for it. I just don't think it should be at the expense of the community. Numbers 3 and 5 sound like nice options that benefit the community and NativeScript.

[Plamen5kov]

@sitefinitysteve

Don't know much about android flavors to know why this matters though :)

You can read more information about the build variants and flavors here. Basically, if developers want to build a free-demo-x86 version of their application containing only the necessary code for those "flavors" they would specify three flavors:

• "demoOrPayed" with dimentions "payed", "free"
• "fullOrDemo" with dimentions "demo", "full"
• "architecture" with dimentions "x86", "arm64", "arm"

But I also don't know much about android studio to even do this, that was always the beauty of the plug-in structure... I didn't need to.

We recognize that not all plugin authors would like to have Android Studio installed and work with it. Fortunately, you wouldn't need to use Android Studio if you don't like to build your plugin. We've devised a workflow that would do everything automatically so it wouldn't change the plugin authors way of doing things.
To be more specific if you're using the plugin seed, you wouldn't need to do any additional steps while developing.

---

@TheOriginalJosh

The proposed changes sound like they will de-incentivize plugin development for Android by making it a little more cumbersome. I understand why the proposed changes are being made and see the need for it.

Thank you for sharing your opinion and know, we hear your concern. That's why we urge plugin developers to use plugin seed so their workflow wouldn't have to be hindered because of possible changes.

I just don't think it should be at the expense of the community.

We strive to bring the best workflow for the plugin authors and for the developers using them. Unfortunately what @NathanaelA says "Works as normal." for the plugins as they're now is partially true, but it doesn't share the whole story. Currently, we're disabling the people using your plugins to also use flavors, which is a common practice while publishing your application. I wouldn't call this normal since we strive to enable as many features of what's possible with the native development workflow.

---

On proposal 3,5:

• builds working not as fast as they can, option 3(5) would suggest plugins that need to be build might take up to 4-5 seconds each.
• we can not guarantee we'd build the plugins correctly because we're not the plugin author that knows best how to build and pack it

I would urge the plugin authors to read this document again and consider the implications each proposal would lead to.

---

@NathanaelA
Could you help me understand how proposal 5 is different from 3?

---

Thank you, everyone, taking part in this discussion!
Read more infor on flavors and how they're helpfull for the users of your plugins:

• https://developer.android.com/studio/build/build-variants.html
• http://www.vogella.com/tutorials/AndroidBuild/article.html#building_gradle_variants

[NathanaelA]

@Plamen5kov - First of all, I was trying to condense things into point to make it easier for everyone; a lot of us don't have a huge amount of free time to skim this entire issue; the original issue #369, pull #388 and the google docs. I summarized each of the points to make it simpler; but if they have extra time I encourage them to read everything related to the issue.

---

"works as normal" being the plugin author can continue doing it the same way and not have any extra work to do. This is my desired outcome. I'm not opposed to extra work; if it gains the eco-system something; but so far option 1,2, & 4 does not help, and instead each of them hurts the eco-system in some fashion. On Flavors; in the many years I've done mobile work; I've never had a client need them. That doesn't mean they aren't useful; as some people do use them; so fixing this for them, is not a bad reason to proceed. But I don't want the "fix" to cause even worse problems, especially since there are other ways to work around not having "real" flavor supported. So I stand by my works as normal as exactly that. 😀

---

Plamen, I expected better from you than this! We both know this is no where near the full truth also, in my case, I wasn't omitting info to make any proposal 1/3/5 look better/worse. I was omitting it because it was pointless for the overall discussion as I was just trying to SUMMARIZE the pros/cons of each and the info excluded had little to no impact on the overall decision making.

builds working not as fast as they can, option 3(5) would suggest plugins that need to be build might take up to 4-5 seconds each.

The FULL truth here, is that the VERY FIRST TIME you build the app after installing/resetting the platform; you MIGHT add a couple extra seconds per plugin, depending on what they are, ram, cpu and hard drives. My understanding is this is NOT really any slower than it is already. And under normal day to day usage; should be the same over all. The only negative here, which I listed as the CON (see I didn't skip it!) is cloud building could be impacted because they build from scratch every time. Your general developer will install the plugin and have a one time hit (like they do now). This speed point really is totally immaterial to the whole discussion other than for Progress and ME personally. (I don't know of any other cloud building providers of NativeScript app, so they would also apply if they exist).

we can not guarantee we'd build the plugins correctly because we're not the plugin author that knows best how to build and pack it

We both know that it is exactly the same under 1 as it would be under 3 or 5. If I have a really weird plugin that must be compiled first (for whatever reason, I can't really think of); there is NOTHING stopping me as I can already do in the current design, or in the proposed option 3, or option 5 from pre-compiling an AAR file and shipping it in my plugin.

Both of these two items are really immaterial to the entire discussion. So lets try not to muddy the waters. The whole idea of my summary and vote was to help consolidate the RELEVANT info to make it clearer. :-)

---

Now as to the differences between 3 and 5. The differences are actually more internal in how it is done. The reason I prefer 5, is it is actually properly using Gradle system properly (after all part of this, is to eliminate using Gradle improperly). Option 5, the plugin is being attached as a sub-project. Having it as a sub-project is actually a lot safer and long term supportable (in my opinion) than some magic in the CLI that has to pre-build each plugin into an AAR first. This should eliminate (a lot?) of extra code in the CLI as then Gradle is doing almost all the work. And finally when you open the final project in Android Studio it is easily modified INCLUDING all the plugins stuff. Which is far superior to even option 1, let alone 3. Where option 3 hides parts of it behind CLI magic and makes modifying the plugins more complex and error prone as the CLI has to maintain the compiled AAR state properly. The Android Studio support, is way better supported under 5 than all of the other options, and if done correctly could even potentially allow us to edit the plugins in studio and have the application sync with the latest version automatically. So that is why 5 is different than 3, and imho a far superior solution all the way around to all of the other proposals.

[PeterStaev]

@NathanaelA

we can not guarantee we'd build the plugins correctly because we're not the plugin author that knows best how to build and pack it

We both know that it is exactly the same under 1 as it would be under 3 or 5. If I have a really weird plugin that must be compiled first (for whatever reason, I can't really think of); there is NOTHING stopping me as I can already do in the current design, or in the proposed option 3, or option 5 from pre-compiling an AAR file and shipping it in my plugin.

From all the talks in this thread and on Slack from the core team seems the whole idea of this proposal is that they want to bring down to the minimum support/issues going to the core/runtime teams because of bad plugins. They want the plugin dev to have a compiled AAR so they are sure that the native things (manifest/resources/code) in the plugin can correctly be compiled so it does not fail when the end user does tns run android. That's where the diff between 1 and 3/5 comes in. I can't say that I don't understand them, but also I don't see this as some big improvement that will benefit everyone.

Also for completeness here is the draft migration steps for Prop1 (shared by Peter over slack)

[NathanaelA]

I haven't ran into bad plugins in that sense. In the normal case if it builds on my machine in the proper format; gradle will build it on your machine. I can't recall the last time a team member had an issue with "bad" plugins that could have been solved by pre-building. Pre-building isn't going to solve any of the issues that I've seen or any that I can think of off the top of my head or any that I've read about in the repo. They might be correct, there might be an issue that I don't recall, but even if it fixes an issue -- I personally believe bad plugins would increase under option 1.

There are two classes of errors that would be created that are only currently an occasional pain right now. MinSDK issues; and Manifest options that can't be overridden properly with tools:replace or tools:ignore. By shipping compiled manifests you increase these types of errors exponentially as now all plugins have this compiled in. Really simple to fix these if it isn't compiled. If this issue (or others related to manifest issues) occurs, the issue will be first posted on the NS group; then NS will say you need to go to the author; the author will then post a support message in the plugins template, because they believe it was a plugins template issue. Wow, that fixed support didn't it. :grinning: We ended up with three different issues, and the final issue still is on Progress to help fix. Support didn't lessen for Progress, it increased as Progress support had to read and respond to TWO separate issues for the same thing. And for an issue that in a lot of cases wouldn't have existed if it wasn't forcing pre-build aar files.

Second the majority of the support issues I've seen in the repos (related to plugins) are normally bad computer setup/configurations. I honestly can't think of a support issue I've seen in the last couple months where it was a bad gradle file or a faulty manifest/resource in an uncompiled plugin.

Thanks for the draft steps; I hadn't seen them. Those are considerably more complicated than I expected. ;-( Good news ( for me 😀 ) based on the draft steps above, is option 1 will drive more paid plugin work to me, so I guess option 1 can't be considered all bad news....

[EddyVerbruggen]

Hey folks, leaving my $0.02 on the subject (of possibly changing the Android plugin structure).

Reading all issues, docs, discussions, whatnot, my opinion is:

It seems that:

• Everybody wants as little as possible impact on current workflows (of course!).
• We all want NativeScript to succeed (why else would you read this at all?).
• We're lucky this project is Open Source, otherwise, we would just get a new version without having a say at all.
• The NativeScript team needs help and wants help on this complex topic, but it's hard for everybody to sync on everything.
• The initial communication could have been better because (also to me) it wasn't abundantly clear that the team is actively looking into automating the migration.

The reasons for changing the Android build are:

• Maintenance debt. Having worked in a few bigger companies and projects I know that's a real problem. Mostly upper management ignores the problem completely until there's a point that a project becomes unmaintainable and you need to start from scratch to be able to actually add new business logic. Without a budget for maintenance, developers start applying patch upon patch and build proprietary tools to keep everything working. I'm happy to learn things like these are flagged and acted upon with NativeScript.
• Allow developers to support multiple flavors in the way Android intended. I don't have this need and I doubt I'll ever will, but I'm sure there are folks wanting to support fi. a Free and Paid version of their app in this way. Especially former native Android developers would appreciate the feature.
• Oh, and I often open a NativeScript project in Xcode to run and debug stuff. Doing this in Android Studio is a big fail because plugins screw up the project (because of the flavors, as I learned).

Going forward, there are a few options, but _ideally_ these criteria should be met:

• If plugins need to be updated, they need to be backward compatible with {N} < 4.
• Works with or without the official NativeScript plugin seed.
• No maintenance of an Android library project.
• No impact on cloud build times.
• If you need something funky, additional work is OK.. but the 90-95% case should be covered.

Best option* IMO: Treat plugins as gradle projects:

• PRO: No changes to the current src/platforms/android structure.
• CON: Slower cloud builds (~4s per plugin that has static files in src/platforms/android), which we have to take seriously because we're not only wasting CPU cycles but also, you have to wait longer for your build to complete (time === money, folks!).
• *POSSIBLE SOLUTION: So I only see this as the 'best option' if there is a way to cache the generated .aar artifacts in a central cloud build repository (not on the internet, but on the Telerik infrastructure, accessible to all build servers) so build times are only marginally affected.

Second best option IMO: plugins generate aar files

• PRO: No changes to the current src/platforms/android structure.
• CON: To be {N} 4 compatible, the plugin needs to be updated as the npm package should contain an aar. Note that with the aar added (and fi. an AndroidManifest.xml and res folder removed) the plugin will still be {N} 3 compatible as it currently already works with aar files just fine.
• POSSIBLE SOLUTION: As having/creating/maintaining a library project is alien to most developers, we could add a "nativescript-plugin-platform-artifact-generator" devDepencency to the plugin's src/package.json. The ultimate goal would be: just keep the plugin as-is, add the devDependency, upon build this new generator takes whatever relevant is in src/platforms/android, and generate an aar for it (with an intermediate Android library project that's cleaned up when the aar is built).

If you have any ideas that may help materialize these proposals or another one that also meets the criteria above, please don't hesitate to spill them!

Hmm, at my rate this was actually more than $0.02 🤔

[NathanaelA]

So has the team decided which method to implement yet; seems like time is running out to implement and test if 4.0 is going to be released by mid-late Feb.

[Plamen5kov]

Hi @NathanaelA,

We're working towards a beneficial solution for the community. Keeping in mind what the people have voted for, we're leaning toward option 3, because it's similar to option 5, but we think we can make it more reliable inside the CLI.

We'll answer this issue as soon as we have something we can provide for the plugin authors to test out. We're working on a proposal for plugin development keeping in mind the preferences of the community. What do you think?

[etabakov]

Just to mention that we are not aiming to release 4.0 mid-late Feb. The Release Candidate for 4.0 will be early-mid March. We also intent to leave the RC more time than usual so that everyone can try out the new bits.

[Plamen5kov]

@NathanaelA, @PeterStaev, @sitefinitysteve, @TheOriginalJosh
Hi guys, sorry it took so long for us to respond. Know that you have been heard and we've taken the appropriate steps to alleviate your pain. Please take a look in this issue and let's continue the discussion there.

[Natalia-Hristova]

Changes in the latest version of android-runtime 4.0.0 can be seen in the CHANGELOG.md