Bazel: Add support for specifying --bazelrc multiple times

[burkpojken]

Description of the problem / feature request:

Today it is not possible to add multiple --bazelrc options to the command line.

Feature requests: what underlying problem are you trying to solve with this feature?

We have some default options in a common bazelrc file that is used in many workspaces since the options are needed to build for our platform.
Today we need to add those options in all user .bazelrc WORKSPACE files.
If it would be possible to add multiple --bazelrc options on the command line we could have one rc file for the common parameters and still be able to use other bazelrc files for the user projects.

What operating system are you running Bazel on?

linux

What's the output of bazel info release?

0.20.0

Have you found anything relevant by searching the web?

No

Any other information, logs, or outputs that you want to share?

[burkpojken]

Are there any objections to this?
We can try to implement it and make a pull request.
Today you do not even get an error or warning if you specify multiple --bazelrc

[laszlocsomor]

Bazel should at least warn that it ignores --bazelrc flags. I'll add that warning.

I opt for a warning instead of an error, because adding an error would be an incompatible change while adding a warning isn't.

(A few months ago the Bazel team decided to adhere to our own incompatible change policy. Adding a new error is an incompatible change. Per policy I had to add an --incompatible_... flag that turns the warning into an error, then flip the flag in the next incompatible release. With our planned 3 month long stability windows, this means 3 months to roll out the flag by default, which seems too long for such a small change as an error message.)

[laszlocsomor]

@burkpojken : Did you know bazelrc files can include other bazelrc files? Could everyone's user rc file, or every project's project-rc file include the common rc file?

[burkpojken]

HI!
I am aware of that you can use include but you cannot use @ to point out another workspace e.g.
We have some flags added from a bazelrc file in our "platform" workspace that then are needed in our "application" workspaces. I really think that it would be useful to be able to use more than one --bazelrc files. Why do you think that would be a bad thing?

[laszlocsomor]

Why do you think that would be a bad thing?

I never said it was. I don't know why Bazel supports just one user-rc file.

Current behavior: multiple --bazelrc flags allowed, first one used rest are ignored, no warning.

I'm looking at adding a warning, but it takes longer than I expected because the code is complex.
I'm not looking at adding an error, because that's an incompatible change.

[laszlocsomor]

Do you mean loading .bazelrc files from repositories declared in the WORKSPACE? I don't think we can support that. .bazelrc parsing is one of the first things Bazel does right after startup, way before even looking at the WORKSPACE.

[egechir]

Hello @laszlocsomor ,

We are starting to work on a feature that requires this behavior and we are from the same workgroup as @burkpojken . We understand the need for this behavior and would like to answer the following question.

Do you mean loading .bazelrc files from repositories declared in the WORKSPACE? I don't think we can support that. .bazelrc parsing is one of the first things Bazel does right after startup, way before even looking at the WORKSPACE.

The intention of this feature request is not to specify multiple .bazelrc files in terms of WORKSPACE, rather it is to specify via the path to the .bazelrc file on the bazel build command.
For example, bazel build --bazelrc=/repo/platform/file1 --bazelrc=/repo/application/file2 .
In this case, the resulting configuration should include options from both the /repo/platform/file1 and /repo/platform/file2. If there is a conflict in the value of an option between those files, then the values from the second file should be taken by bazel.

As you have stated previously, the current behavior is that the first file is used and the rest is ignored. I believe this is the behavior we want to change. We hope that it won't be a completely new behavior because today it is possible for options placed in different .bazelrc files to override each other.

Do you think this is a desirable feature for Bazel upstream ?
If so, would you reopen this issue so that we can continue the discussions ?

[laszlocsomor]

Hi @egechir ,

Do you mean loading .bazelrc files from repositories declared in the WORKSPACE? I don't think we can support that. .bazelrc parsing is one of the first things Bazel does right after startup, way before even looking at the WORKSPACE.

The intention of this feature request is not to specify multiple .bazelrc files in terms of WORKSPACE, rather it is to specify via the path to the .bazelrc file on the bazel build command.

Thanks for clarifying. That makes sense to me, but it seems to be different than @burkpojken's request:

I am aware of that you can use include but you cannot use @ to point out another workspace

If I understand correctly, this is about referencing workspaces either in bazelrc files or in --bazelrc flags, which is infeasible with the current design.

Do you think this is a desirable feature for Bazel upstream ?

Hard question.

I understand your use case and I think it'd be a nice feature. But I don't think it's desirable _for upstream_. It would be if many people requested the feature and implementing it benefited many, but that doesn't seem to be the case.

There's another reason I'm hesitant to consider implementing or upstreaming this feature: I suspect certain Google-internal logging infrastructure to depend on --bazelrc having a single value, so upstreaming the change would (if my hunch is right) require changing that infrastructure, which I wouldn't do unless there's internal demand, which there seems to be none.

The workaround is to have a single bazelrc file and import other bazelrc files. I understand it's not as flexible as multiple --bazelrc flags, but it is available and it works.

If so, would you reopen this issue so that we can continue the discussions ?

Sure. If you wanted to implement this feature, I'd happily review the PR and run it against internal tests and tell you of any complications.

[egechir]

Thanks for the reply @laszlocsomor .
We will make an estimate of a possible implementation and will get back to you if we intend to make a pull request. So, the issue need not be reopened now.

[laszlocsomor]

Thanks! (And sorry I didn't reopen when I said so, forgot it like I never wrote it..)

[emlrsua]

I have a question. I'm working with egechir on this issue. I've tried changing the "bazelrc" definition in BazelStartupOptionsModule.java so that it has "allowMultiple = true" and so that the field is now a "List". I'm wondering where in the code it will use this data so that I can make it accept and pass on a List rather than a single String. Do you have any advice there?

[laszlocsomor]

@emlrsua : It's not used from Java. The flag is only defined in Java to attach documentation to it: https://github.com/bazelbuild/bazel/blob/d9fe1d4078c25912427c9b2e79e115f4ee9b7ab8/src/main/java/com/google/devtools/build/lib/bazel/BazelStartupOptionsModule.java#L35

The actual flag handling is in C++, here: https://github.com/bazelbuild/bazel/blob/d9fe1d4078c25912427c9b2e79e115f4ee9b7ab8/src/main/cpp/bazel_startup_options.cc#L46

[wisechengyi]

Do you think this is a desirable feature for Bazel upstream ?

Hard question.
I understand your use case and I think it'd be a nice feature. But I don't think it's desirable for upstream. It would be if many people requested the feature and implementing it benefited many ..

Hi @laszlocsomor,

Please kindly reopen the issue if you think the case is valid, and redirect if you are no longer active on bazel.

I'd like to raise this again on behalf of Twitter, multiple --bazelrc on CLI would be useful for us for various reasons:

1. Mostly importantly, we will have very long argument lists to pass to bazel query, so they have to be in a bazelrc file. From the info above, import/try import in bazelrc would still work but very awkwardly. For example, if there are multiple bazelrc files to import, say A and B,
   
   
   
   1. import A needs to be added into$WORKSPACE/.bazelrc
   
   
   2. import B needs to be added into A
   
   

meaning the former bazelrc file needs to know what comes next.

1. Allowing multiple --bazelrc would be similar to how we configure Pants run in CI today, because CI needs to pass different configs so we use --pants-config-files=a,b,c,d where a,b,c,d are different config files and the latter overwrites the former, so the concept of --bazelrc would translate directly.

The reality is, per https://docs.bazel.build/versions/master/guide.html#bazelrc-the-bazel-configuration-file, multiple bazelrcs are allowed in some order of WORKSPACE bazelrc, homedir bazelrc, etc, so I think it would make sense to allow it on CLI as well.

[laszlocsomor]

@meisterT could you please re-route

[ulrfa]

Thanks for reopening @meisterT. This would be useful also for Ericsson, for the same reasons mentioned by @wisechengyi.

[philwo]

I'm in favor of supporting this and can't think of good reasons not to. 🤔

The concerns that Laszlo had earlier about backend systems (e.g. logging, monitoring, ...) assuming that --bazelrc is a singular option might be annoying if true, I agree with that. But as you all said, Bazel already supports multiple sources of bazelrcs already anyway, so the concept of having multiple files at least isn't new.

Would someone like to contribute a PR? My team would be happy to review and merge.

[wisechengyi]

Happy to take a stab at this. Hoping to have something out by EOY, but if anyone wants to jump in prior, please do by all means.

[wisechengyi]

Review out at https://github.com/bazelbuild/bazel/pull/12740