Organizations in the Maven world leverage the BOM concept to predefine "blessed" or "known" dependency versions. Enable Gradle to consume BOM natively.
Gradle users can define dependencies without a concrete version. The version is resolved from a BOM that's defined for the build. The build fails if the version is not defined for the group
and name
of the dependency. User do not have to use external plugins anymore.
Gradle does not support the consumption of BOMs. Users that want to consume BOMs have to consume external plugins e.g. the Pivotal dependency management plugin.
Make it easier for projects that migrate from Maven to Gradle to reuse existing Gradle infrastructure.
As the main developer of Pivotal's dependency management plugin, I'd love to see it become obsolete. If there's anything I can do to help, please do just let me know
@wilkinsona I'd say a good way to start on this would be to 1) start a discussion on the Gradle dev-list and 2) write a spec. Your input would be super valuable given that you have a lot of experience with the support of BOMs in the Gradle. Would you be interested in kicking off a discussion on the dev-list?
User do not have to use external plugins anymore.
What's the issue here? The fact that a community plugin could add this functionality on top of our APIs tells me that they are sufficient to solve all kinds of interesting use cases. I think we should get away from binding ourselves to Maven instead of adding even more of its concepts into Gradle core.
As the main developer of Pivotal's dependency management plugin, I'd love to see it become obsolete.
Could you elaborate why that is? If the maintenance of that plugin is hard, then we should find out what the problems are and help with those. I'd be very much in favor of keeping this in a community plugin, making that plugin efficient and easy to maintain and then promoting it for anyone who misses BOMs. E.g. gradle init
could automatically apply that community plugin when it converts a Maven project that was using a BOM.
Could you elaborate why that is?
It's largely subjective, but it feels to me like the sort of thing that Gradle should support natively. Maven is the de facto standard for dependency metadata on the JVM and boms are becoming increasingly popular (Jackson recently added one, for example). Many users of the plugin have expressed their thanks for the plugin and their surprise that it was needed.
If the maintenance of that plugin is hard, then we should find out what the problems are and help with those
It's certainly caused some problems in the past, but things appear to be better now. The plugin's now 100% Java (it was written in Groovy previously) and doesn't try to support such a broad range of Gradle versions. Both those changes seem to have helped.
Many users of the plugin have expressed their thanks for the plugin and their surprise that it was needed.
I think we want to get away from the sentiment that "using/needing a community plugin" is somehow bad/surprising. The plugin ecosystem is what makes Gradle great. I'd much rather move more stuff out of Gradle's core distribution than pulling things in. Especially if there already is a dedicated and capable maintainer. This will require more work on our side of course, e.g. stabilizing APIs, making core plugins less "special" and promoting community plugins.
This will require more work on our side of course, e.g. stabilizing APIs, making core plugins less "special" and promoting community plugins.
Irrespective of the decision here, this is something that I'd really like to see. I won't retread old ground too much here, but if Gradle's core plugins only used public API then it'd go a long way to levelling the playing field. Perhaps that's part of what you meant by making them less special?
Yes, that's exactly what I mean.
Thanks for opening the discussion on the dev list, @wilkinsona. Looks like it didn't get any traction over there and the discussion is rather happening here.
My 2 cents about this topic: I agree that we need to have a better set of public APIs so that plugin developers don't need to use internal APIs. I also agree with @wilkinsona that Maven BOM support feels very core. Gradle prides itself with being almost 100% compatible with Maven - but then doesn't support an important feature of the Maven spec. We do support other portions of that same spec like consumption of parent POMs, Maven properties etc. That sounds kind of weird and makes it hard for people migrating from Maven to have a smooth transition.
In the long run we could think about Gradle's Maven support as a separate module or plugin for Gradle but we are long ways away from that. Either way I see users run into issues with BOM consumption all the time. It is frustrating and doesn't shine a good light on Gradle. I am thinking of Gradle as an "integrator" of other tools. From that perspective I don't see why we shouldn't support BOM consumption natively. Yes, it will require effort, work and future maintenance on our end but right now we are rather at risk of losing users or at the very least confusing/frustrating them.
The use of BOMs is core to our maven infrastructure. I'm surprised this isn't considered a necessary feature of Gradle resolution since its the Maven idiomatic way to address dependency consistency across multiple projects. Yes, we can accomplish this using nebula-recommender or pivotal plugins but then we have to rely on Nebula/Netflix or Pivotal for their plugin updates and they may not have the Gradle road map or ideology close in mind (slack in updates) nor do they necessarily have the same capacity or priorities.
Maybe this comment may be a little off topic, but kind of within the same topic. I was just thinking why limit to maven BOMs?
In case that one needs to support both maven and gradle, then it will most likely be advantagous from a maintenance perspective to support a single format and that would most likely be BOMs, however in case that one has gone all in with gradle. IMHO then maybe it would be better to use a format that was more gradle like.
So maybe rather than have some logic integrated to the gradle core and get locked into maven BOM. I think it would be more beneficial to have an API for resolving dependencies and get the flexibiltiy to resolve dependencies using maven BOMs or some alternative.
I think it would be more beneficial to have an API for resolving dependencies and get the flexibiltiy to resolve dependencies using maven BOMs or some alternative.
And that's exactly what Gradle's dependency resolution infrastructure already provides. We can certainly make that more powerful, so that BOM support becomes easier to implement, faster etc. But I wouldn't hard-bake BOMs into Gradle's core.
BOM support feels quite core for me too. Most of our JEE projects use a BOM to determine what dependencies play well together.
But the solution of this issue could be something in the middle: continue using a plugin but treat it as "official".
I rather prefer to have this functionality in the core but "official" plugins with gradle support could be an option
For anyone looking for the commit to see the changes c1076008bb70a9bb2dd4ad62c6a6ba4014def375
I'm really pleased to see Gradle getting native support for Maven boms. Thank you for adding it.
Looking at the commit that @rwinch referenced above, I didn't notice any support for controlling the value of a property that's used in a bom. For example, Spring Boot's bom uses properties to control the versions of dependencies that it manages directly and boms that it imports. This allows users to easily override a version. Have I missed something in the changes made from Gradle 4.4 or is this something that could be considered as a further enchancement?
@wilkinsona I'd consider that a further enhancement, as it applies to any other Maven dependency too (i.e. we don't support setting properties and we only have very limited support for profiles).
I'd consider that a further enhancement, as it applies to any other Maven dependency too
AFAIK, that's the not case. Maven only supports overriding properties when a (p|b)om is used as the project's parent.
Yes, but since Gradle has no concept of a "parent", we'd need some other channel to convey properties to the Maven resolver. Probably on a project-wide basis, not per dependency. Does that make sense?
It does. As shown in the link I shared above, that's exactly what the Dependency Management Plugin does. Unfortunately, without the ability to override properties, Gradle's bom support will be of limited use to Spring users and we'll continue to recommend the use of the dependency management plugin instead.
I think we'll solve that another way with the upcoming constraints DSL. As far as I can see spring.version
only affects dependencies with a particular group. This could be managed using something like (pseudo code ahead!):
constraint {
group 'org.springframework'
version '4.0.4.RELEASE'
}
This would upgrade the spring version to 4.0.4 even if the BOM contained something lower.
The major upside of this would be that these constraints would be propagated to downstream projects (both inside a multi-project build and through publishing to a repository). Maven resolver adjustments like setting properties on the other hand would be confined to a single build and thus not very useful when you have downstream consumers, because they would see the unaltered BOM.
I think we'll solve that another way with the upcoming constraints DSL
I think it would be better to support a Maven-specific concept in a way that maps directly onto Maven's way of doing things. To do differently would repeat the problems with subtly different handling of Maven exclusions.
As far as I can see
spring.version
only affects dependencies with a particular group
That's true for spring.version
but it's by no means guaranteed to be the case, particularly when the property is controlling the version of a bom that's imported. A concrete example is Spring Boot's bom which imports Jetty's bom. Jetty's bom manages the versions of dependencies in multiple different groups.
With some regret, I have to say that I think this improvement in Gradle is actually going to make things worse overall. In my opinion, for this functionality to be generally useful, it needs to support as much of what Maven can do with a bom as possible, and do so in the same way as Maven. If the functionality is incomplete or works in subtly different ways to Maven's own support for the same concept, users are going to be left wondering how to map something that's easy when using a bom in Maven to Gradle and will sometimes come up blank. They'll then also have to decide between Gradle's own bom support or one of the third-party plugins that's currently filling the gap.
Does Maven even support overriding properties in an imported BOM? Afaik it only supports overriding properties in the parent POM, which is a different use case.
Does Maven even support overriding properties in an imported BOM?
As long as you use the bom that defines the properties as your parent, you can override its properties. That means that you can override the version of a bom that's imported by your parent bom. It's only when you import a bom directly in your own pom that you lose the ability to override its properties.
which is a different use case
Why do you consider that to be a different use case? In terms of dependency management, I think the use case is the same: use a bom to manage the versions of some dependencies.
It's clear that "BOM support" is a pretty large topic, and we were premature in declaring this feature as "delivered". We haven't communicated well enough about the BOM support that we _are_ working on, which is being developed as part of a larger "better dependency management" project.
To clarify, the work we've done is the equivalent of BOM _import_ into a Gradle build. A Gradle build will be able to declare a dependency on a BOM module, which will add version constraints to the build as well as any downstream consumers. This should allow Gradle users to leverage the spring-boot
BOM file in the same way Maven users can via import
.
(Note that this will be a minor breaking change to dependency resolution, and so will initially be guarded behind a feature flag.)
What we have _not_ done is any work to allow a Gradle build to mimic the use of a BOM _as a parent POM with property overrides_. I think this is what @oehme was referring to as a "different use case".
What I'd like to do is reopen this issue and convert it into an Epic. This will allow us to add sub-issues for specific elements of BOM support and we can discuss and prioritize them individually.
It’s great to see this available (behind a feature flag) in 4.6 RC1. With things progressing, is there any news on the sub-issues for this now-epic issue? I’m particularly interested in properties support as this is something that’s widely used by Spring Boot’s users and, without it, I remain concerned about the confusion that will be caused as users try to figure out what to use.
It would be cool to have some use cases for that. You've probably seen most of them.
To me at first glance it seems that properties would water down the meaning of a BOM. I.e. a BOM says "these things work well together". If I start changing that with properties, I'm basically voiding that warranty.
Also afaik Maven does not support properties for BOMs. I.e. if you use the import
scope, that imported BOM will not have its properties replaced. Afaik property replacement only works with parent poms.
In stable release semantics, a patch release ahead of any of the BOM defined dependencies should be compatible. BOM sometimes requires large horizontal synchronization of release schedule and does not tie everything in an exclusive project lifecycle. Some modules will be updated and available prior to larger BOM release, it happens to us quite often with our Reactor BOM.
So the use case would be "I want to upgrade part of the platform defined by this BOM to a newer version". As a user, why don't I just add a direct dependency to the newer thing I want?
There are a few cases where it's useful to be able to override a version using a bom's property:
We see all three of these regularly being used by Spring Boot's users.
Also afaik Maven does not support properties for BOMs. I.e. if you use the import scope, that imported BOM will not have its properties replaced. Afaik property replacement only works with parent poms.
That's correct, and it's something that Maven users regularly complain about. The dependency management plugin removes this restriction when using a bom in Gradle and it's been warmly welcomed by users. There's been some discussion about improving the situation in Maven but the pace of change on that side of the fence is rather slow.
As a user, why don't I just add a direct dependency to the newer thing I want?
Because that would defect the purpose of using a bom in the first place. If the library contains multiple modules, potentially spanning multiple different group IDs, it's far easier to set a single property to pick up the new version than it is to add versions to all your relevant dependences or to write a resolution strategy that applies to all the relevant group IDs.
Let me play devil's advocate a little further here: If the thing I want to upgrade itself consists of multiple libraries that need to be aligned together, why does that part not have its own BOM?
It often does. When that’s the case, Spring Boot will import the bom into its own bom, using a property to control its version. Users can then override the property to change the library’s version. This provides a consistent experience for overriding versions, irrespective of whether or not the library has a bom of its own.
To me it looks like there already are solutions: Well-behaved libraries have a BOM, which you can depend on in order to upgrade. Badly-behaved libraries can be fixed with component metadata rules. Or someone else can publish a BOM for that library if the original author can't be convinced.
Is your concern mostly around convenience of using Gradles APIs to fix versioning problems?
Adding support for properties feels like a redundant feature to me. It adds some convenience, but only for a limited use case. Properties are only a thing in Maven. Gradle's metadata format does not have properties.
We also plan to have alignment constraints which will allow you to concisely say "I need 5.0 for everything in the org.foo group". At that point even the convenience difference for badly-behaved libraries goes away.
I understand that properties are the only knob that Maven offers, but Gradle already has a feature set that solves these use cases and we need to be careful not to add more features that do the same thing. We don't plan to mirror Maven features 1:1 but rather adapt it to Gradle's model as best we can.
That's my view at least, I'm sure @jjohannes and @bigdaz also have some ideas around this. And of course I'm curious about your take on the comments above.
Does adding a BOM like below include all the dependencies specified in the pom.xml.
dependencies {
// import a BOM
implementation 'org.springframework.boot:spring-boot-dependencies:1.5.8.RELEASE'
// define dependencies without versions
implementation 'com.google.code.gson:gson'
implementation 'dom4j:dom4j'
}
I would expect a clear difference from declaring dependencies when specifying a BOM.
Something like:
dependencyManagement {
importBom 'org.springframework.boot:spring-boot-dependencies:1.5.8.RELEASE'
}
That also implies some properties from the pom.xml
A very useful way to manage consistent dependencies and properties across multiple projects.
To me it looks like there already are solutions: Well-behaved libraries have a BOM, which you can depend on in order to upgrade. Badly-behaved libraries can be fixed with component metadata rules. Or someone else can publish a BOM for that library if the original author can't be convinced.
Is your concern mostly around convenience of using Gradles APIs to fix versioning problems?
That's certainly part of it.
Another part of it is that publishers of a bom that is regularly used as a parent design that bom expecting it to be possible to override the properties that it uses. That's an entirely reasonable expectation for those publishers to have, as support for properties when used as a parent is an integral part of Maven's bom support.
Whenever Gradle adopts a Maven concept and consumes Maven metadata to support that concept, confusion arises if the default behaviour is different or certain features are not supported. We've seen it in the past with subtly different handling of optional dependencies declared in pom files (the requirement to be declared optional on all edges of the dependency graph vs a single edge). I think ignoring properties for bom imports would be making a similar mistake.
Another problem with not supporting properties is that the bom becomes a leaky abstraction. A consumer of a bom shouldn't have to know how it manages the versions of a particular library's modules when they want to override a version. If a property can be set to override the version, a user gets the same experience irrespective of how many group IDs the library's modules use and irrespective of whether the versions are managed directly in the bom or by a transitive import of a bom.
I understand that properties are the only knob that Maven offers, but Gradle already has a feature set that solves these use cases and we need to be careful not to add more features that do the same thing. We don't plan to mirror Maven features 1:1 but rather adapt it to Gradle's model as best we can.
Maven offers another knob. You can declare dependency management in your own pom and it will override the dependency management in the parent or in an imported bom. This is equivalent to importing a bom in Gradle and then declaring some component rules to override the bom. I haven't seen anyone use this approach in situations where they can use a property instead.
Thanks for the conversation, we really appreciate the effort you take in explaining your use cases and why you think things should work one way or another. You and our users deserve answers and I think we need to give a bit more context to understand our decisions and reflect on whether they are correct or not. First of all, as @oehme explained, our goal is not to map 1:1 to Maven concepts. We will when we think it makes sense, and we will propose different solutions when we think there are better ways to model things.
Our medium term goal is to:
Support for BOMs fall into the "convenience" category: this is a widely used metadata format found in the Maven ecosystem, that we had to support one way or another. This is finally happening in 4.6 and we must rejoice. However, we also have our plans, and they involve understanding what library authors, what platform maintainers and what users all think when they use BOMs.
There are different use cases for BOMs, the main ones being:
The reality is that people do not use BOMs for the same purpose everywhere, and reading questions on StackOverflow makes you realize things are far from perfect. Part of the problem with BOMs is that the intent and the implementation do not match (in the Maven world). Since they are not specified, the behavior is determined by the implementation, that you have to retro-engineer. This may be fine, but it's an illustration of something that was built ad-hoc, and that we use "because it kind of works".
For example, the technical definition of what a BOM is is only determined by the kind of dependencies that are found in a POM file. The file format is the same, and it's the presence, or absence, of both dependencyManagement
or dependencies
block that determine the semantics of the POM file. It may be a good idea to share the same metadata format, but we don't believe that having different semantics based on what you find in a file is a good idea, especially when you don't know if the BOM is going to be used as a "parent POM" or an "imported BOM". This, for us, means that we need to do best effort to support BOMs from the Maven world, but that we also have to give better solutions to our users and move on.
One particular problem with Maven and BOMs is that BOMs are not enforced transitively. If you, as a platform maintainer, write a BOM, it's most likely to express that "those things work together, please make sure the versions you choose are compatible". However, BOMs do not make any guarantee. The dependency management blocks are used:
This means that a library that imports a BOM will potentially build, publish and pass QA, but once it's integrated, the BOM is not enforced. This is terrible behavior, we can surely do better. In short, it matters more to us that users get _understandable_ and _predictable_ results, than mimic’ing the behavior of Maven (you have to be aware that Maven POM files is only one of the metadata options that Gradle supports, but we effectively support more, like Ivy and potential custom models).
This is why 4.6 ships with the concept of "dependency constraints". Unlike BOMs, dependency constraints ARE transitive. If you say something about a dependency, if you say those things work together, they need to be enforced for the full graph (which also means we can potentially fail the build if you have conflicts). This is only a first milestone in the long series of features we're going to ship in 5.0. We want to solve use cases, not "simulate BOM". We cannot live on a weaker publishing model anymore.
So let's illustrate with the properties case. When you say "users can override the property to use a newer version of a library", there are a lot of hidden assumptions in there:
So what can we do if we want to upgrade a dependency which is expressed using a property? One option is just to add an explicit dependency:
implementation 'org.springframework.boot:spring-boot-dependencies:1.5.10.RELEASE'
implementation ('com.fasterxml.jackson:jackson-bom:2.9.4') {
because “we depend on features added in Jackson 2.9” // overrides the version 2.8.10 from the Boot POM
}
And you might say “what if the version is shared by different modules?”. This is a legit case for properties in Maven. However, this is not how we want to solve that. @oehme explained that what we will provide, instead, is a way to express that “those modules are aligned”, be it by group or something else. In other words, we want to model that “those things work together”, rather than relying on a weakly defined property, which _may_ do the work, if there’s no conflict in the graph. By modeling, we will make sure it actually happens, instead of accidently working.
To summarize, this means that we’re committed to supporting the Maven semantics of a published BOM as much as we can, but we might end up not supporting every feature Maven adds on top exactly the same way. Instead, we will provide alternative solutions to the use cases that fit the Gradle model and we’re focused on making this happen before or for 5.0 (alignment, fixing bad MD, ...). We will update this issue to discuss these solutions further once they are ready. So, it’s not because this issue is closed that it means we’re done with BOMs ;)
Hi Ben, can you please give me feedback if you still work on this issue? We are using spring-dependencies-plugin facing a performance issue and I am at the point to decide to add an issue at springboot-dependencies-plugin or to use the bom feature coming with gradle. So would be nice if you could give me some info on when this feature will be released
@moley This topic is still on our radar although recent releases have not seen a change to what was delivered with Gradle 4.6 behind the feature flag IMPROVED_POM_SUPPORT
. That also means there is no target Gradle version thus far, though 5.0 is seriously considered.
@moley If you have time, can you please file an issue for your performance problem with the Dependency Mangagement Plugin anyway? If you have a project that can reproduce the slowness, I'll happily take a look and see what can be done to speed things up.
ok, thanks for your info, gradle 5.0 will come this year or later?
@wilkinsona We have a really big project with many transitive dependency layers, so it's not that easy. But I am trying at the moment to create one with deps from mavencentral and open an issue.
@moley You can use the Gradle profiler to create flame graphs for @wilkinsona, so he can see the bottlenecks without having your build. E.g. run gradle-profiler --profile jfr --warmups 10 --iterations 40 assemble
to measure the performance of a fully up-to-date build
Hi Stefan, thanks for this suggestion, but I saw to late. Meanwhile I have created a testproject, which I have uploaded to github. I think we have a problem with both gradle dependency resolution and spring-plugin resolution, because even with gradle it lasts 30s (with spring 1 min 15 s). Should I open one issue per project?
We're always happy to get performance data, please open an issue :)
@wilkinsona @oehme
I added two issues, because the gradle dependency resolution itself is slow itself and spring-gradle-plugin increases the problem:
https://github.com/spring-projects/spring-boot/issues/13979
https://github.com/gradle/gradle/issues/6171
Feature will be part of Gradle 5.0
Most helpful comment
Thanks for opening the discussion on the dev list, @wilkinsona. Looks like it didn't get any traction over there and the discussion is rather happening here.
My 2 cents about this topic: I agree that we need to have a better set of public APIs so that plugin developers don't need to use internal APIs. I also agree with @wilkinsona that Maven BOM support feels very core. Gradle prides itself with being almost 100% compatible with Maven - but then doesn't support an important feature of the Maven spec. We do support other portions of that same spec like consumption of parent POMs, Maven properties etc. That sounds kind of weird and makes it hard for people migrating from Maven to have a smooth transition.
In the long run we could think about Gradle's Maven support as a separate module or plugin for Gradle but we are long ways away from that. Either way I see users run into issues with BOM consumption all the time. It is frustrating and doesn't shine a good light on Gradle. I am thinking of Gradle as an "integrator" of other tools. From that perspective I don't see why we shouldn't support BOM consumption natively. Yes, it will require effort, work and future maintenance on our end but right now we are rather at risk of losing users or at the very least confusing/frustrating them.