Bazel: Integrate external dependencies with their native build system

[meteorcloudy]

When migrating projects to Bazel, users often have to migrate their third party dependencies to Bazel. This is usually a significant amount of work which could slow down migration process a lot. See envoy's migration story: https://github.com/bazelbuild/bazel/issues/2792

To help our users, Bazel could provide some Skylark rules that invoke the native build systems of the dependencies, then provide the outputs as targets that can be depended on by other targets.

This method is worthy as it will help our users migrating to Bazel more easily. An experimental rule for cmake/configure&make is planned in Q2 2018 with productionisation in Q3

Related docs:
TensorFlow Bazel Build Issues
Bazel External Dependencies Roadmap

[meteorcloudy]

@dslomov As the engineer who's going to work on this is leaving the team, can you find a proper assignee for this issue?

[jin]

Also see: https://github.com/bazelbuild/rules_foreign_cc

[meteorcloudy]

@jin Thanks, Jingwen!!!

[jbduncan]

I'm not sure if I'm overstepping my boundaries here, but I wanted to share my thoughts about this issue.

On top of migrating from CMake for a C++ project, as #2792 discusses, this would be brilliant for other build systems, such as Gradle in my case.

I've been interested for a while now for a solution that would allow me to build my Gradle-based vanilla Java projects with Bazel (mainly so that I have a reason to learn and play around with Bazel :wink:), but also to be able to continue using certain Gradle plugins and tasks that I've grown dependent on (e.g. Spotless, gradle-dependencies-plugin). I've also been hoping for an incremental solution, so that I can take my time to fully migrate things to Bazel if I so wish.

I was hoping that the OkBuck Gradle plugin would allow me to do this, as some time ago @aj-michael proposed some initial support for Bazel (https://github.com/uber/okbuck/issues/232), but sadly at the time of me writing this that issue seems to have petered out.

So if there's anything I can do to help move this issue forward, then I would love to be involved. :)

[jin]

@jbduncan You might be interested in Spotify's https://github.com/menny/bazel-leaf, which lets you incrementally build Gradle modules using Bazel.

[jbduncan]

Thanks @jin! I'll have a proper look at it when I'm next free. :)

[jbduncan]

@jin AFAICT, bazel-leaf doesn't seem to have any installation instructions yet. Am I missing something? :)

[jin]

@jbduncan It looks like there are setup instructions here: https://github.com/menny/bazel-leaf#module-setup

[jbduncan]

@jin Hmm... I think they're incomplete, because the second instruction,

• apply the bazel-leaf plugin:

apply plugin: 'bazelleaf'

...assumes that one has already imported the plugin somehow, which the setup instructions don't seem to say anywhere.

If one just adds the above line to their build.gradle file, it errors out with "Plugin with id 'bazelleaf' not found".

In my experience, that means that one needs to import it from the Gradle Plugins Portal, which the bazel-leaf README doesn't seem to mention anything about. A quick search over at the Gradle Plugins Portal for "bazel", "bazelleaf", "bazel-leaf" and "blaze" apparently doesn't turn up anything related... :thinking:

[jbduncan]

@jin FYI, I've just raised an issue over at bazel-leaf: https://github.com/menny/bazel-leaf/issues/3.

[jin]

@jbduncan Ah, I see. I am not familiar with how you can specify bazelleaf as a local Gradle plugin. Perhaps there is a way to do that?

On Jun 11, 2018, at 9:10 PM, Jonathan Bluett-Duncan [e-mail] wrote:

@jin Hmm... I think they're incomplete, because the second instruction,

apply the bazel-leaf plugin:
apply plugin: 'bazelleaf'
...assumes that one has already imported the plugin somehow, which the setup instructions don't seem to say anywhere.

If one just adds the above line to their build.gradle file, it errors out with "Plugin with id 'bazelleaf' not found".

In my experience, that means that one needs to import it from the Gradle Plugins Portal, which the bazel-leaf README doesn't seem to mention anything about. A quick search over at the Gradle Plugins Portal for "bazel", "bazelleaf", "bazel-leaf" and "blaze" apparently doesn't turn up anything related... 🤔

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or mute the thread.

[jbduncan]

@jin Yep, I'm sure there is. I will either have to compile the plugin locally on my machine and store the jar in a lib directory in my project, or use jitpack.io. Not terribly ideal, but hey ho - for experimentation purposes, it should work. :)

[kageiit]

Sidenote, Okbuck is migrating to skylark for all generated build file definitions in buck - https://github.com/uber/okbuck/issues/598

okbuck is going to use custom rules i.e disallow native rules in the generated BUCK files. It could be a lot easier potentially to tweak the loads for the .bzl files that define these rules and make them use the bazel native rules instead underneath. This would make inter-op a lot easier (buck specific config in the rules can just be ignored or converted over in pure skylark)

This might be an easier path to support bazel via okbuck

[jin]

@kageiit thanks for the update! The Bazel Android rules are not implemented in bzl yet (https://github.com/bazelbuild/rules_android), but it'll be interesting to see if Bazel will be able to use these new .bzl files for Buck.

[bbarnes52]

"An experimental rule for cmake/configure&make is planned in Q2 2018 with productionisation in Q3"

Is this timeline still accurate?

[petroseskinder]

@bbarnes52
Might want to check with:
https://github.com/bazelbuild/rules_foreign_cc

[jbduncan]

I'm not entirely sure if this issue is intended to be resolved by the time Bazel 1.0 is released. (I understand from https://blog.bazel.build/2019/06/06/Bazel-Semantic-Versioning.html that Bazel 1.0 is intended to be a "milestone" for implementing the Bazel Vision, and that implementing this issue is part of the vision.)

Could someone clarify this for me? :)