Redex: Support for Android Bundles?

[awelc]

It does not seem that ReDex supports Android Bundle format at this point. At least when I try to redex a a bundle APK file, I get an error in the unpacker.py script when trying to detect secondary dex mode. Upon closer inspection, it seems like the problem is that the detect method used to figure out the secondary dex mode searches for dex files directly in the directory where APK is unpacked, whereas in case of bundles these would be in the base subdirectory.

I was wondering if someone could comment on whether Android Bundles are supported (behind an option perhaps) and, if not, if there is a plan to support them.

It seems like perhaps it only requires changes to the unpacker.py script, but a comment on that would be very much appreciated as well.

[emma0303]

Thank you for bringing this to our attention.

For the moment, they aren't supported. We will look into it, though we will probably not get to it in the near future. If you need this feature faster, feel free to send a PR and we will review it.

[awelc]

Let me re-open this in the hope that I can get feedback on how to get the bundle support to the next stage and support dynamic feature modules. In one of the comments in https://github.com/facebook/redex/pull/390 @justinjhendrick has said:

_"In this initial version we're only looking at the base dexes, but the additional modules can have any arbitrary names, right? How do we discover all the modules? How do we make sure that redex optimizes each module separately? I think there's already a concept of a DexStore. Let's make sure we wire these modules in as separate DexStores correctly."_

I am not sure that I got this right, but I think that DexStores are currently used to represent individual dex files. In Android bundles, each module can have multiple dex files, so I am not sure if a 1:1 mapping between DexStores and modules can be made.

At the same time, perhaps we can simply map all dex files from all modules to DexStores (singling out the 1st dex file of the base module much like we currently do for the primary dex file of the whole APK).

It seems like the dependency between modules goes only one way - other modules can depend on the base one, but not the other way around (nor can they depend on one another as they can be installed/downloaded separately). This is also implied by in the Android Bundle documentation (https://developer.android.com/studio/projects/dynamic-delivery) stating that we are dealing with the dependency tree (rooted at the base module) rather than with the dependency graph:

_"Note that the base APK forms the head of the tree, and all other APKs depend on the base APK"_

Perhaps it would then be OK to pull all these dex files together (much like we do for "regular" APKs) and also analyze them together. This seems to make sense for optimizations like inlining, but I am not sure how general this would be. I can see some problems already, but perhaps they can be worked around. For example, it's not super clear what we'd do with InterDex, but perhaps because this optimization is aimed at startup improvements, it's OK to apply it to only dex files in the base module.

I would appreciate any and all feedback on this before starting the actual work on this.

[justinjhendrick]

DexStores are currently used to represent individual dex files

I don't think this is true in general.

A DexStore represents a module. There can be multiple dexes in a DexStore. Though sometimes we treat the primary dex as its own module because of older phones that don't have native multi-dex support.

For example, an app could have 6 dexes in 4 DexStores:
1) classes.dex [1]
2) secondary-1.dex, secondary-2.dex
3) feature_bar.dex
4) feature_foo.dex, feature_foo2.dex

See CheckBreadCrumbsPass for an example usage of DexStores. It looks for "illegal" cross store references which can cause crashes.

[1] classes.dex isn't actually in its own module, but redex treats it that way out of convenience