Bazel: Add Android data binding support

[gregestren]

Bazel already has internal support for Android data binding (e.g. here). But hooks haven't been exposed to link this to the SDK support libraries or toggle the feature on for android_binary / android_library rules.

This bug tracks the effort to make data binding end-user usable.

[gregestren]

Context: https://groups.google.com/d/msg/bazel-discuss/AOO6T6UpTRY/y74pENzmBAAJ

[gregestren]

Update: we're making internal progress on this change and can start exposing further hooks in Bazel likely over the next week. This still won't be functional though without having the data binding libraries checked into appropriate places in your repo.

Anyone interested in experimenting with this earlier, and willing to play around with checking in the data binding libraries themselves, report here and we're glad to work through the process with you.

[aj-michael]

Update: After a discussion with @gregestren we've determined that we need to update the aar_import rule to handle the databinding metadata files that are included in the com.android.databinding:adapters:1.3.*:aar libraries.

[Paul-McNeice]

It looks like there has been various commits in relation to this issue since the last comment - just wondering how close we are to having databinding support up and running?

[jin]

@ahumesky

[gregestren]

Paul,

As Jin referenced above, @ahumesky is the current point of contact for data binding support. He's working through a number of technical issues about it at the moment and can give you the clearest expectations.

[ahumesky]

We're working to upgrade databinding in bazel to the latest version to support androidx / jetpack, as well as make depending on databinding easier. We've also found some bugs with databinding + android_instrumentation_test. We hope to have these issues sorted out in the next couple of weeks.

[Paul-McNeice]

Hey @ahumesky, any update on the status of this?

[ahumesky]

This was interrupted by a few little urgent issues, but those are done and we're back to work on this. I hope to have v2 working this week, then I plan to make getting the databinding dependencies working with the android rules easier (today you have to manually wire up the databinding dependencies).

[codesuki]

Hey, I am trying to convert a project to Bazel using databindings. Is there any progress on this? Even a hacky workaround would help me a lot until it's officially working.
When enabling databinding on android_binary Bazel crashes with NPE.

[ahumesky]

There has been significant progress, I have databinding v2 working, and I'm in the process of getting all the code reviewed and submitted. Should be in in the next couple of days.

You're seeing an NPE probably because the databinding dependencies need to be added to the rules with databinding enabled (it shouldn't give an NPE for that though, that's a bug in itself). I'm also working to make getting the databinding dependencies easier. The main problem right now is that our gmaven repo for android dependencies is incomplete (databinding depends on things that are hosted outside gmaven). It would be better to use something like bazel-deps to generate the rules for these dependencies, but bazel-deps does not support AARs.

[codesuki]

Thanks for the update. That sounds good!

I tried adding

aar_import(
    name = “library-1.3.1”,
    aar = “@androidsdk//:extras/android/m2repository/com/android/databinding/library/1.3.1/library-1.3.1.aar”,
)

As a dependency but it had no effect. Probably it's wrong anyway.

Do you have a sample on how to add the dependency correctly? I could adapt that to my local installation. Right now I don't even know which library is the correct one. And the only one I found is the one in the SDK.
😅

[codesuki]

This is the NPE when setting enable_data_binding = True .
Bazel versions 0.15.2 and 0.18.1.

Internal error thrown during build. Printing stack trace: java.lang.RuntimeException: Unrecoverable error while evaluating node '//:app BuildConfigurationValue.Key[c46469ce7e7c9c01ec016ebdb5b40fd0] false' (requested by nodes )
    at com.google.devtools.build.skyframe.AbstractParallelEvaluator$Evaluate.run(AbstractParallelEvaluator.java:495)
    at com.google.devtools.build.lib.concurrent.AbstractQueueVisitor$WrappedRunnable.run(AbstractQueueVisitor.java:368)
    at java.util.concurrent.ForkJoinTask$AdaptedRunnableAction.exec(ForkJoinTask.java:1386)
    at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289)
    at java.util.concurrent.ForkJoinPool$WorkQueue.execLocalTasks(ForkJoinPool.java:1040)
    at java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1058)
    at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1692)
    at java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:157)
Caused by: java.lang.NullPointerException
    at com.google.devtools.build.lib.rules.java.JavaCompilationHelper.usesAnnotationProcessing(JavaCompilationHelper.java:465)
    at com.google.devtools.build.lib.rules.android.AndroidCommon.initJava(AndroidCommon.java:606)
    at com.google.devtools.build.lib.rules.android.AndroidCommon.init(AndroidCommon.java:540)
    at com.google.devtools.build.lib.rules.android.AndroidBinary.init(AndroidBinary.java:251)
    at com.google.devtools.build.lib.rules.android.AndroidBinary.create(AndroidBinary.java:119)
    at com.google.devtools.build.lib.rules.android.AndroidBinary.create(AndroidBinary.java:95)
    at com.google.devtools.build.lib.analysis.ConfiguredTargetFactory.createRule(ConfiguredTargetFactory.java:319)
    at com.google.devtools.build.lib.analysis.ConfiguredTargetFactory.createConfiguredTarget(ConfiguredTargetFactory.java:205)
    at com.google.devtools.build.lib.skyframe.SkyframeBuildView.createConfiguredTarget(SkyframeBuildView.java:635)
    at com.google.devtools.build.lib.skyframe.ConfiguredTargetFunction.createConfiguredTarget(ConfiguredTargetFunction.java:777)
    at com.google.devtools.build.lib.skyframe.ConfiguredTargetFunction.compute(ConfiguredTargetFunction.java:323)
    at com.google.devtools.build.skyframe.AbstractParallelEvaluator$Evaluate.run(AbstractParallelEvaluator.java:418)
    ... 7 more

https://github.com/bazelbuild/bazel/blob/0.18.1/src/main/java/com/google/devtools/build/lib/rules/java/JavaCompilationHelper.java#L463

I tried setting javacopts to include -processor just to see if the NPE goes away but nothing changed.

[cre8ivejp]

@ahumesky Hi
Any progress on this issue?

[ahumesky]

The current state of things is:

1. Bazel supports databinding "v1", however v1 has performance issues, and also requires all the databinding dependencies to be manually wired up.
2. Bazel has been updated to support databinding "v2" which should solve the performance issues, however we ran into some bugs in bazel / databinding integrations while trying to roll this out internally that will require fixes on the databinding side. I hope to have those integrated in the next week or two. v2 integration also requires the databinding dependencies to be pulled in in a particular way, so in the meantime I'm working to make that smoother.

[cre8ivejp]

Thank you for your reply.
For the v2 integration, I think as long we can make this work, I would be happy to pull the dependencies in that particular way :)
Let's pray to everything goes well and we can start using from next month

[act262]

Is there any progress now?

[zachgrayio]

Hey there - just checking in and see what the status is on this as lack of databinding support is a big blocker for us on adopting bazel in the manner we'd like to.

Is there anything we can do to contribute to this effort?

[mfiels]

@ahumesky any new updates on where the v2 databinding support is currently at in bazel? Is there any way the community can help? would be great to see this supported!

[changusmc]

Would love an update on the progress of Data Binding support, I'm getting an NPE similar to other folks in this issue. Our MvX architecture relies on Data Binding.

[roscrazy]

Hi guys. It is 2020, I believe this is one of the main blockers for many teams to adopt Bazel
Do we have any updates?

[zackify]

I've tried everything to get databinding to work, its weird that enable_data_binding seems to only "work" on android_binary. (I say work in quotes because the command will run, but databinding still doesnt work at all) But if you add it to android_library you get the error above from 2018 https://github.com/bazelbuild/bazel/issues/2694#issuecomment-437185740

why do the docs say this is supported when it doesnt work? Even went through and added all the databinding dependencies from maven, but nothing appears to work

[ahumesky]

Hi all,
If you're interested in trying databinding integration, it's a little easier now with the third-party dependencies checked in.

1. Get the bazel source git clone [email protected]:bazelbuild/bazel.git bazel
2. cd bazel
3. bazel build //src:bazel //tools/android/runtime_deps:android_tools.tar.gz
4. Extract the android tools somewhere:

mkdir /tmp/android_tools
tar -xf bazel-bin/tools/android/runtime_deps/android_tools.tar.gz -C /tmp/android_tools

1. Create tools/android/BUILD with this in it:

java_plugin(
    name = "compiler_annotation_processor",
    processor_class = "android.databinding.annotationprocessor.ProcessDataBinding",
    generates_api = True,
    visibility = ["//visibility:public"],
    deps = [
      "@bazel_tools//src/tools/android/java/com/google/devtools/build/android:all_android_tools",
    ],
)

1. Add the databinding app dependencies, and a bind rule to find the annotation processor plugin added above:

load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive")

RULES_JVM_EXTERNAL_TAG = "2.4"
RULES_JVM_EXTERNAL_SHA = "2393f002b0a274055a4e803801cd078df90d1a8ac9f15748d1f803b97cfcdc9c"

http_archive(
    name = "rules_jvm_external",
    strip_prefix = "rules_jvm_external-%s" % RULES_JVM_EXTERNAL_TAG,
    sha256 = RULES_JVM_EXTERNAL_SHA,
    url = "https://github.com/bazelbuild/rules_jvm_external/archive/%s.zip" % RULES_JVM_EXTERNAL_TAG,
)

load("@rules_jvm_external//:defs.bzl", "maven_install")

maven_install(
    artifacts = [
      "com.android.databinding:adapters:3.4.2",
      "com.android.databinding:library:3.4.2",
      "com.android.databinding:baseLibrary:3.4.2",
      "com.android.support:support-annotations:28.0.0",
    ],
    repositories = [
        "https://jcenter.bintray.com/",
        "https://maven.google.com",
        "https://repo1.maven.org/maven2",
    ],
)

bind(
  name = "databinding_annotation_processor",
  actual = "//tools/android:compiler_annotation_processor",
)

1. Add the databinding app dependencies to the app, e.g.:

load("@rules_jvm_external//:defs.bzl", "artifact")

package(default_visibility = ["//visibility:public"])

android_binary(
    name = "app",
    custom_package = "com.google.android.app",
    enable_data_binding = 1,
    srcs = glob(["src/main/java/**/*.java"]),
    resource_files = glob(["src/main/res/**/*"]),
    manifest = "AndroidManifest.xml",
    deps = [
      artifact("com.android.databinding:adapters:3.4.2"),
      artifact("com.android.databinding:library:3.4.2"),
      artifact("com.android.databinding:baseLibrary:3.4.2"),
      artifact("com.android.support:support-annotations:28.0.0"),
    ],
)

1. Build using the bazel and android tools built above, and with some other flags:

~/bazel/bazel-bin/src/bazel build app --strategy=Desugar=sandboxed --android_databinding_use_v3_4_args --experimental_android_databinding_v2 --override_repository=android_tools=/tmp/android_tools

The --strategy is to work around #11618. Once the fix for that is in, we can deploy a new version of the android tools, and then neither that nor the --override_repository flags will be needed. --android_databinding_use_v3_4_args and --experimental_android_databinding_v2 can also be defaulted to true now, so in a future version those won't be necessary either. (v1 never worked with bazel, and the pre-v3.4 args no longer work either). The java_plugin() at tools/android/BUILD can probably be moved into the remote android tools BUILD file, and there's probably a way to get rid of the bind.

[ahumesky]

One correction: the java_plugin in the build file at tools/android/BUILD needs generates_api = True. I've updated the comment above.

[arunkumar9t2]

Thanks @ahumesky, the suggestions work so far for us. Do you have any plans to add support for androidx?

EDIT: Looks like it is being worked on here: https://github.com/bazelbuild/bazel/commit/f4430e4e7ec9e832e2b6d0e7f1c108256563bd23

EDIT2: Can confirm the above commit works with androidx