Gradle: Deploying timestamped snapshots to Maven 2 repositories does not produce/update the maven-metadata.xml files properly

Created on 6 Sep 2017  路  46Comments  路  Source: gradle/gradle

Hi,

I have originally posted this on Stackoverflow, but I am becoming more and more convinced that this looks like a bug in Gradle.

I have the following build.gradle script:

apply plugin: 'java'
apply plugin: 'maven'

compileJava {
    targetCompatibility = "1.8"
    sourceCompatibility = "1.8"
    options.encoding = 'UTF-8'
}

ext.username = System.getProperty('username') != null ?
               System.getProperty('username') : mavenUser
ext.password = System.getProperty('password') != null ?
               System.getProperty('password') : mavenPassword

description = "Example project"

configurations {
    deployerJars
}

repositories {
    maven { url "http://nexus:8081/nexus/content/groups/my-group/" }
    mavenCentral()
}

dependencies {
    deployerJars "org.apache.maven.wagon:wagon-http:2.2"
    testCompile group: 'junit', name: 'junit', version: '4.11'
}

// Gradle build Configuration to publish artifacts to maven artifact manager
uploadArchives {
    repositories.mavenDeployer {
        configuration = configurations.deployerJars
        repository(url: "http://nexus:8081/nexus/content/repositories/my-snapshots/") {
            authentication(userName: username, password: password)
        }
        pom.groupId = "com.foo.bar"
        pom.artifactId = "my-gradle-app"
        pom.version = "1.0-SNAPSHOT"
    }
}

task sourcesJar(type: Jar, dependsOn: classes) {
    classifier = 'sources'
    from sourceSets.main.allSource
}

task javadocJar(type: Jar, dependsOn: javadoc) {
    classifier = 'javadoc'
    from javadoc.destinationDir
}

// add javadoc/source jar tasks as artifacts
artifacts {
    archives sourcesJar
    archives javadocJar
}

When I execute the following, the build and deployment succeeds:

$ gradle clean upload -Dusername=myUsername -Dpassword=myPassword

The produced maven-metadata.xml file looks like this:

<metadata>
    <groupId>com.foo.bar</groupId>
    <artifactId>my-gradle-app</artifactId>
    <version>1.0-SNAPSHOT</version>
    <versioning>
        <snapshot>
            <timestamp>20170906.143113</timestamp>
            <buildNumber>1</buildNumber>
        </snapshot>
        <lastUpdated>20170906143113</lastUpdated>
    </versioning>
</metadata>

These are the files that have actually been deployed to Nexus:

my-gradle-app-1.0-20170906.143113-1-javadoc.jar
my-gradle-app-1.0-20170906.143113-1-javadoc.jar.md5
my-gradle-app-1.0-20170906.143113-1-javadoc.jar.sha1
my-gradle-app-1.0-20170906.143113-1-sources.jar
my-gradle-app-1.0-20170906.143113-1-sources.jar.md5
my-gradle-app-1.0-20170906.143113-1-sources.jar.sha1
my-gradle-app-1.0-20170906.143113-1.jar
my-gradle-app-1.0-20170906.143113-1.jar.md5
my-gradle-app-1.0-20170906.143113-1.jar.sha1
my-gradle-app-1.0-20170906.143113-1.pom
my-gradle-app-1.0-20170906.143113-1.pom.md5
my-gradle-app-1.0-20170906.143113-1.pom.sha1

Coming from a Maven background, I am expecting to see the following in the maven-metadata.xml file:

<metadata>
    <groupId>com.foo.bar</groupId>
    <artifactId>my-gradle-app</artifactId>
    <version>1.0-SNAPSHOT</version>
    <versioning>
        <snapshot>
            <timestamp>20170906.143113</timestamp>
            <buildNumber>1</buildNumber>
        </snapshot>
        <lastUpdated>20170906143113</lastUpdated>
    </versioning>
    <snapshotVersions>
        <snapshotVersion>
            <classifier>javadoc</classifier>
            <extension>jar</extension>
            <value>1.0-20170906.143113-1</value>
            <updated>20170906140847</updated>
        </snapshotVersion>
        <snapshotVersion>
            <classifier>sources</classifier>
            <extension>jar</extension>
            <value>1.0-20170906.143113-1</value>
            <updated>20170906140847</updated>
        </snapshotVersion>
        <snapshotVersion>
            <extension>jar</extension>
            <value>1.0-20170906.143113-1</value>
            <updated>20170906140844</updated>
        </snapshotVersion>
        <snapshotVersion>
            <extension>pom</extension>
            <value>1.0-20170906.143113-1</value>
            <updated>20170906140847</updated>
        </snapshotVersion>
    </snapshotVersions>
</metadata>

If I re-run the deployment, I would be expecting the maven-metadata.xml file to no only have the <snapshot/>'s <timestamp/> and <buildNumber> to be updated, but also to get new <snapshotVersion/> entries appended. What's happening instead is, the whole file is regenerated and there's just one <snapshotVersion/> and only the <snapshot/>'s <timestamp/> and <buildNumber> are correctly updated.

I have tried this with Gradle 2.13 and 4.1.

I am very familiar with the inner workings of the Maven Metadata format and I have actually documented it here, should you find it useful.

What am I missing here? Why isn't the maven-metadata.xml being produced properly? Is there a plugin that I am missing? Is this indeed a bug in Gradle?

bug contributor publishing-plugin

Most helpful comment

Hi,

Apologies for my late reply! Been caught up in other tasks.

Like I was saying my earlier in messages, Maven 2.2.1 reached an end of life on 2009-11-08, as described here. That's just about 8 years ago. The question here is: how much of a tolerance period do you need to be giving before announcing that you no longer support it? I understand that you would like to be as backward-compatible, as possible, but at some point you need to drop old stuff in favour of more modern improvements.

I don't have time to investigate in what way this may affect ancient versions of Maven such as 2.x, but what I can tell you for sure is that it's more important to be compatible with modern day versions of tools. If you;d like to be backward-compatible, give it 2-3 years (five...?!), anything above that would be bad for everyone. If you keep supporting ancient technology, it will never die and people will never move forward and improve things.

Gradle will continue to issue HEAD requests to determine the existence of classified artifacts, and will ignore the element. I really don't understand the purpose of listing each snapshot artifact, since the same indexing information isn't published anywhere for non-SNAPSHOT artifacts, AFAIK.

In a way I might agree that this is only for snapshots, but... If this is how it works in the Maven world and you are using and producing Maven dependencies (after all), this is what you need to follow. Like it, or not... If you would like to discuss this further, I would recommend contacting the Maven Developers distribution list.

Will Maven2 clients still be able to correctly consume repositories generated with the extended maven-metadata files?

I would say "no", although I have not tried it.

Does Maven provide tooling for cleaning up stale SNAPSHOTs, including updating maven-metadata.xml? Having historic SNAPSHOT versions in the metadata means that this will no longer be a simple matter of removing the old files.

This would not normally be handled by Maven. Artifact repository managers have cron tasks that can be configured to keep the last X number of built snapshots and clear up the older ones. When they do this, they take care of the maven-metadata.xml files.

In conclusion, I would like to recommend that you consider this potentially breaking change as part of your next release. You can make announcements to get people informed... It appears that the actual fix will not involve too much work, according to what @bigdaz has described.

Looking forward to hearing your thoughts!

Martin

All 46 comments

Thanks for reporting and I've confirmed this behaviour. Looks like at some point between 2.12 and 2.13 this behaviour had been changed according to your description and this post.

Since I'm not an expert on Maven, I have a question, what's the purpose of <snapshotVersions> tag and what's the impact of not having this tag?

Hi,

Thanks for getting back to me!

In Maven snapshots work like this...

(A bit of history, for a clearer picture) In Maven 2.x there were two types of snapshots:
* Timestamped (otherwise known as unique versions) - where 1.0-SNAPSHOT would actually be represented by 1.0-20170908.100515-1. There would be no physical file called foo-1.0-SNAPSHOT.jar, but foo-1.0-20170908.100515-1.jar, where 1.0 is the snapshot's base version, 1.0-20170908.100515 is the timestamp and 1 is the build number. Build tools such as Maven, Gradle, SBT, Ant+Ivy, etc. would then download the maven-metadata.xml file and figure out what the latest timestamped snapshot version is and construct the URL to it based on this. So, for example, when downloading it's dependencies Maven would first resolve these maven-metadata.xml files and generate proper URL-s to the respective timestamps.
- Non-timestamped snapshots. These would be just 1.0-SNAPSHOT the file would exist physically on the file system and be overwritten when there are new deployments.

In Maven version 3 the non-timestamped snapshots were dropped. (Maven 2.x was EOL-ed a few years ago).

If you have a look at the page I included in my post, you can see a very detailed explanation of the [Maven Metadata|https://github.com/strongbox/strongbox/wiki/Maven-Metadata] format.

There are three types of Maven Metadata:

  • Artifact version-level : This is used for releases and snapshots. It lists the available versions (1.0, 2.0, 1.0-SNAPSHOT, 2.0-SNAPSHOT. etc).
  • Snapshot version-level : This is used for describing the timestamped snapshot versions (for example for 1.0-SNAPSHOT it will have the latest <timestamp/> and <buildNumber/>, as well as a list of all the artifact files belonging to this artifact (a.k.a. sub-artifacts; for example: pom, jar + jars for classifier-based artifacts, such as sources, javadocs and so on, if any)).
  • GroupId level: This is only relevant for Maven Plugins and from Gradle's point of view, it should not be a concern, unless somebody is developing a Maven plugin and using Gradle to build and deploy it (which I think would be a highly-unlikely scenario).

A valid maven-metadata.xml for a timestamped snapshot artifact which has been deployed twice would look like this:

<metadata>
    <groupId>com.foo.bar</groupId>
    <artifactId>my-gradle-app</artifactId>
    <version>1.0-SNAPSHOT</version>
    <versioning>
        <snapshot>
            <timestamp>20170908.123705</timestamp>
            <buildNumber>2</buildNumber>
        </snapshot>
        <lastUpdated>20170908123705</lastUpdated>
    </versioning>
    <snapshotVersions>
        <snapshotVersion>
            <classifier>javadoc</classifier>
            <extension>jar</extension>
            <value>1.0-20170906.143113-1</value>
            <updated>20170906140847</updated>
        </snapshotVersion>
        <snapshotVersion>
            <classifier>sources</classifier>
            <extension>jar</extension>
            <value>1.0-20170906.143113-1</value>
            <updated>20170906140847</updated>
        </snapshotVersion>
        <snapshotVersion>
            <extension>jar</extension>
            <value>1.0-20170906.143113-1</value>
            <updated>20170906140844</updated>
        </snapshotVersion>
        <snapshotVersion>
            <extension>pom</extension>
            <value>1.0-20170906.143113-1</value>
            <updated>20170906140847</updated>
        </snapshotVersion>
        <snapshotVersion>
            <classifier>javadoc</classifier>
            <extension>jar</extension>
            <value>1.0-20170908.123705-2</value>
            <updated>20170908123705</updated>
        </snapshotVersion>
        <snapshotVersion>
            <classifier>sources</classifier>
            <extension>jar</extension>
            <value>1.0-20170908.123705-2</value>
            <updated>20170908123705</updated>
        </snapshotVersion>
        <snapshotVersion>
            <extension>jar</extension>
            <value>1.0-20170908.123705-2</value>
            <updated>20170908123705</updated>
        </snapshotVersion>
        <snapshotVersion>
            <extension>pom</extension>
            <value>1.0-20170906.143113-2</value>
            <updated>20170908123705</updated>
        </snapshotVersion>
    </snapshotVersions>
</metadata>

One thing to pay attention to is that the timestamp for the artifact must be the same for all of it's described sub-artifact files for the build number. It should not be off by a second, or two, as this can break things. I believe I've already seen this issue with Gradle as well.

From what I'm seeing across the internet, people seem to be relying on the artifact repository manager to rebuild the Maven metadata for them. (If they have a scheduled task for this). This is not the correct way of doing things, as it's the responsibility of the build tool to be taking care of this. Also, if they were to rely on this, who knows when the scheduled task would kick in... It might be hours before this happens (usually this is scheduled once every 24 hrs, especially in large organizations with hundreds of repositories, as this operation can take quite some time to finish when executed against an entire repository).

The problem is that when you deploy the artifacts with Gradle to a Maven 2 repository in Nexus, other build tools might have dependencies on these artifacts. In our case, we have a third-party continuous deployment product which expects valid Maven metadata in order to resolve the snapshot artifacts and then deploy them to the respective environments. Teams are resorting to the really unorthodox approach of building their code with Gradle and then deploying it with Maven using the mvn deploy:deploy-file goal, just so that they would have properly generated Maven Metadata.

Generating proper Maven metadata is also required in order to produce proper Maven indexing (which I've roughly described here). This is produced by artifact repository managers and is used by IDE-s and other artifact repository managers. This is how your IDE knows which class is coming from which dependency.

So, basically, by not properly supporting this there are many things that would be affected.

Could you guys look into this, please?

Sure. Thanks for your extremely detailed response! We'll dig into this!

Thanks! Let me know, if you need any help with further explanations. I will do my best to help with that!

Unfortunately, I am not as familiar with the Gradle codebase, as I am with Maven, so I don't think I'll be able to prepare a pull for this myself, but feel free to ask, if you have any questions on how this should work.

In addition, I have also tried this with the maven-publish plugin and the result is the same.

I've dug into this a bit and here's my discovery:

This behavior in Gradle is consistent (I've tried 2.7,2.12,2.13,3.5). This is because of a property maven.metadata.legacy here. If I modify it to true, the result metadata would have snapshotVersions tags as you described.

However, I doubt if snapshotVersions tags are necessary in metadata. I published a snapshot with Gradle 4.1 to nexus, and it generates the following metadata:

<?xml version="1.0" encoding="UTF-8"?>
<metadata>
  <groupId>com.foo.bar</groupId>
  <artifactId>my-gradle-app</artifactId>
  <version>1.0-SNAPSHOT</version>
  <versioning>
    <snapshot>
      <timestamp>20170909.131632</timestamp>
      <buildNumber>2</buildNumber>
    </snapshot>
    <lastUpdated>20170909131632</lastUpdated>
  </versioning>
</metadata>

and the following artifacts:
image

So I guess the metadata without snapshotVersions can be accepted by most maven clients - they know repository drops the support of non-timestamped snapshots, so they fetch the metadata, get the timestamp and buildNumber, then concatenate them and request that artifact.

You mentioned this can break things, could you give me an example?

Hi,

Thanks for further investigating this!

I would like to point out that Maven 2.x has been EOL-ed in November 2009. This legacy mode was intended for the expected timeframe during which people would be migrating off Maven 2.x to 3.x. The fact that Maven repositories across the various repository managers are called "Maven 2" is irrelevant. As Maven 2.x has been EOL-ed, it would only make sense to support the current format.

Basically, Gradle claims to be interoperable with Maven and is using Maven repositories for storing it's artifacts. It should also comply with the way Maven works, or how it expects things to be done.

Skipping the <snapshotVersions/> part means that your implementation of Maven metadata is just half-implemented. As explained in my previous reply, It's fine to not implement the metadata for Maven plugins.

The tool we're using is a vendor product by CA called Nolio (this is a continuous deployment application). The Maven project is producing multiple sub-artifact files (jar, pom, javadoc, sources, among others), all produced and deployed using Gradle. If these files are not described as per the Maven Metadata rules, the tool fails to find the timestamped artifacts and, consequently, fails.

What Gradle needs to be doing is this:

  • Not produce the Maven metadata in legacy format, but use the modern-day one (this has been around for eight years now, so it's not something experimental; also there's now talks of planning work for Maven 4, as the 3.5.x releases will be coming to an end).
  • Describe all the artifact's sub-artifact files
  • If a maven-metadata.xml exists in the repository when deploying a new timestamped snapshot, download it and append the new timestamp's artifact/sub-artifact files to the existing metadata and not overwrite it like it does now.
  • Update the <snapshot/> and <lastUpdated/> tags accordingly.

I hope this helps. :)

I totally understand you and thank you for your kind advice. I'll try to change maven.metadata.legacy to false and see if it cause any issues.

@adammurdoch What do you think?

Hi!

Is there any update on this?

@oehme What do you think about this?

@blindpirate I don't have the history on this change. Can you git blame that line and ask someone who was involved?

@oehme Sure, thanks!

@blindpirate ,

Any update? :)

@carlspring Not yet. I'm waiting for @bigdaz 's brilliant opinion on this. @bigdaz I'd like to hear your opinion on this when you're back from your vacation.

Okay, thanks for the update @blindpirate ! :)

@bigdaz , @blindpirate ,

Has anyone had a chance to have a proper look at this? :)

@carlspring thanks for the detailed analysis. Gradle attempts to be interoperable with Maven as much as possible. In priority order, this means:

  1. Produce a Maven repository that can be consumed by Gradle
  2. Produce a Maven repository that can be consumed by many different versions of the Maven build tool. (This includes Maven v2)
  3. Produce a Maven repository that is understood by the major IDEs.
  4. Produce a Maven repository that can be consumed by third-party tools.

In practise, #4 isn't something we have put much effort into, and that seems to be what's biting you here. You mention that IDEs are also suffering from the missing information in maven-metadata.xml: it would be great if you could be more specific on this, so we can fully understand the scope of the problem.

Gradle doesn't directly generate maven-metadata.xml. Instead, we delegate this work to org.sonatype.aether.RepositorySystem here. As @blindpirate mentioned, it's the fact we are setting the maven.metadata.legacy property that's producing the current behaviour.

So in theory, the fix should be simple. However, I'm curious about the implications in general:

  • Gradle will continue to issue HEAD requests to determine the existence of classified artifacts, and will ignore the <snapshotVersions> element. I really don't understand the purpose of listing each snapshot artifact, since the same indexing information isn't published anywhere for non-SNAPSHOT artifacts, AFAIK.
  • Will Maven2 clients still be able to correctly consume repositories generated with the extended maven-metadata files?
  • Does Maven provide tooling for cleaning up stale SNAPSHOTs, including updating maven-metadata.xml? Having historic SNAPSHOT versions in the metadata means that this will no longer be a simple matter of removing the old files.

Once we understand the implications of the change, we can decide if it's best to just change the current behaviour, or if we'll require some more sophisticated modelling to allow the user to specify the target 'version' or the published maven repository.

Hey there. I was curious if there have been any updates on this and/or if there were any workarounds while we still have this current behavior? Thanks.

Hi,

Apologies for my late reply! Been caught up in other tasks.

Like I was saying my earlier in messages, Maven 2.2.1 reached an end of life on 2009-11-08, as described here. That's just about 8 years ago. The question here is: how much of a tolerance period do you need to be giving before announcing that you no longer support it? I understand that you would like to be as backward-compatible, as possible, but at some point you need to drop old stuff in favour of more modern improvements.

I don't have time to investigate in what way this may affect ancient versions of Maven such as 2.x, but what I can tell you for sure is that it's more important to be compatible with modern day versions of tools. If you;d like to be backward-compatible, give it 2-3 years (five...?!), anything above that would be bad for everyone. If you keep supporting ancient technology, it will never die and people will never move forward and improve things.

Gradle will continue to issue HEAD requests to determine the existence of classified artifacts, and will ignore the element. I really don't understand the purpose of listing each snapshot artifact, since the same indexing information isn't published anywhere for non-SNAPSHOT artifacts, AFAIK.

In a way I might agree that this is only for snapshots, but... If this is how it works in the Maven world and you are using and producing Maven dependencies (after all), this is what you need to follow. Like it, or not... If you would like to discuss this further, I would recommend contacting the Maven Developers distribution list.

Will Maven2 clients still be able to correctly consume repositories generated with the extended maven-metadata files?

I would say "no", although I have not tried it.

Does Maven provide tooling for cleaning up stale SNAPSHOTs, including updating maven-metadata.xml? Having historic SNAPSHOT versions in the metadata means that this will no longer be a simple matter of removing the old files.

This would not normally be handled by Maven. Artifact repository managers have cron tasks that can be configured to keep the last X number of built snapshots and clear up the older ones. When they do this, they take care of the maven-metadata.xml files.

In conclusion, I would like to recommend that you consider this potentially breaking change as part of your next release. You can make announcements to get people informed... It appears that the actual fix will not involve too much work, according to what @bigdaz has described.

Looking forward to hearing your thoughts!

Martin

Any update? :)

Thanks for following up and providing even more context. I like the idea of making this change in the next major version of Gradle, so that we can document this as a potentially breaking change. We are already planning some other breaking changes related to Maven compatibility (e.g handle optional dependencies), and we could add this to the list. You can already get a sneak preview of these changes by running Gradle with -Dorg.gradle.internal.experimentalFeatures.

In addition, I think we should change the default for the incubating maven-publish plugin in the next minor release.

Is there some sort of master task linking these things that we can follow? When is this next major release scheduled for?

I've created a PR to produce the correct maven-metadata when using the maven-publish plugin. We have plans to make this plugin stable in Gradle 5.0, and with a possible deprecation of repositories.mavenDeployer. For this reason, I'd like to close this issue once the PR is merged.

@carlspring have you investigated using maven-publish in your build?

When is this next major release scheduled for?

We have tentative plans for Gradle 5.0 mid-2018.

Wow... That's a long time from now... And would you happen to know when the next minor version would be scheduled for?

I believe I had tried the maven-publish as well (a while ago). Would you happen to have a fully working example? (Thanks!) :)

Unfortunately, my fix does not result in the full history of versions in maven-metadata.xml. This is because the library we use to publish relies on the snapshots being present in .m2/repo, and for Gradle this is not (always) the case.

Fixing this would involve a deeper fix for maven publishing.

Any update on this?

@carlspring We've made some improvements to the maven-publish plugin in this regard, but:

Unfortunately, my fix does not result in the full history of versions in maven-metadata.xml. This is because the library we use to publish relies on the snapshots being present in .m2/repo, and for Gradle this is not (always) the case.

While we acknowledge this limitation in our maven-publish plugin, we don't currently have the capacity to schedule a fix for this.

@carlspring Is creating the full history necessary for your tool or was it just failing because of the old format? Creating the full history is more complicated, as we'd have to "fake" an m2 repository containing the current data from the server, so the Maven deployer can merge it. This is not something we'd prioritise unless it is a blocker for many users or someone provides a pull request with the necessary changes.

@oehme,

I believe you're confusing things.

When an artifact is being deployed to the artifact repository manager:

  • If there is no maven-metadata.xml file, a new one should be generated.
  • If there is a maven-metadata.xml file, download it and add the new timestamped snapshot details to it.

If the artifact repository manager contains artifacts which were deployed using Groovy and not cron task for rebuilding the Maven Metadata, then obviously, the maven-metadata.xml will not contain all the records for the artifacts which already exist in it the repository. The build tool should not be concerned with this (although, in Gradle's case, it's the one causing the mess), as the repository manager can sort this out (whether through a manual invocation, or a cron task). The problem with invoking this manually, is that you need to know when to do it and the problem with having it as a cron task is that, it would normally be slow when executed against a real-life large repository.

Consider looking into this situation too:
https://discuss.gradle.org/t/maven-metadata-with-classifiers-fails-to-resolve-for-snapshots/20071

Googling for this problem (https://www.google.es/search?q=Maven+metadata+with+classifiers+fails+to+resolve+for+snapshots) seems to point to several third party tool having such a problem.

Here's a explanation of the maven metadata format: https://github.com/strongbox/strongbox/wiki/Maven-Metadata - linked to earlier too.

This task is well over a year old now... with a clear explanation of both the problem and the solution. Are there any plans to address this properly and completely resolve the issue?

I think there are a bunch of issues mixed up here, so it would be best to split this up into several stories. Here's the current status:

  • The maven-publish plugin has been using Maven 3 metadata for 10 months now
  • The old maven plugin still uses Maven 2, but we can change that in 5.0. However, you're much better off using maven-publish anyway.
  • There were no changes regarding generation of a full history. This hasn't been a priority for us, as we've had very few complaints about this so far. If you'd like to drive this forward, a pull request would be the best way to go.
  • If there are any other issues that I missed during this discussion, we should probably have separate tickets for those too and turn this one here into an epic linking to them.

Hi,

Thanks for the update!

I thought you were downloading the maven-metadata.xml from the remote and then updated it...?

I was basing that purely on @bigdaz previous answer - But after trying it out, it actually seems to work just fine. The aether API automatically downloads the metadata and merges it without intervention from us. I.e. this code comment seems wrong.

I looked into this some more and I guess there was some confusion about two different files here:

  • the top-level maven-metadata.xml for each library will contain all versions ever published in the version block, both releases and snapshots
  • for each version there is another maven-metadata.xml, which can contain a <snapshotVersions> block. This block is not cumulative, i.e. Maven will overwrite it with the latest timestamped version. It only keeps an old entry if something is missing in a new snapshot, e.g. you removed a classified artifact.

The maven-publish plugin does both of these updates correctly. Can you give it a try and let me know if anything is missing?

for each version there is another maven-metadata.xml, which can contain a block. This block is not cumulative, i.e. Maven will overwrite it with the latest timestamped version. It only keeps an old entry if something is missing in a new snapshot, e.g. you removed a classified artifact.

This doesn't seem to agree with the example provided by @carlspring here:
https://github.com/gradle/gradle/issues/2882#issuecomment-328079778

I'll admit to not having tried this out with Maven.

If Maven behaves differently than the aether API we use then there's not much we can do. That would mean they don't use the library they themselves provide.

All documentation and examples I can find online indicate that the versions are overwritten, so I think the example in the second post was wrong. The one in the first post on the other hand looks right, with exactly one version per classifier.

@oehme ,

Would you mind illustrating what's not right in the sample? I'm not sure I follow? Have I perhaps made a mistake?

The second example you gave contains multiple snapshotVersions for the same artifact. This is not possible according to both the code and documentation of Maven. It only keeps the latest snapshotVersion for each artifact, like you showed in your first post.

Just as an example, here is a library that is published using Maven. It has published many snapshots, but there is only one <snapshotVersion> for each artifact. This makes sense, because you only need to know what the latest one is.

@bigdaz @oehme According to the following threads, it appears this feature was actually removed. Is there a reason why that happened?
https://discuss.gradle.org/t/maven-metadata-with-classifiers-fails-to-resolve-for-snapshots/20071
https://discuss.gradle.org/t/gradle-3-1-seems-to-default-to-maven-metadata-legacy-true/20362
If compatibility with Maven isn't a priority, that's fine, but these kinds of important limitations should at least be documented somewhere.

If compatibility with Maven isn't a priority, that's fine, but these kinds of important limitations should at least be documented somewhere.

I can't see how Maven won't be a priority. Gradle doesn't have it's own layout format, as far as I'm aware. It's using Maven's layout format, as well as Maven repositories in artifact repository managers (such as Archiva, Artifactory, Nexus, Strongbox, etc). If it publishes to such repositories, then it must follow the way that they work, as well as the way that tools which will use these repositories will work. You can't simply rely on the artifact repository manager to have a cron task that rebuilds the metadata, because you don't want to implement proper support for it. You have to be a good citizen and integrate well. Otherwise... what's the point?

If you can't produce valid metadata, how will other consumers be able to use it?

I don't understand why this thread is resurrected: the comments above clearly states that using maven-publish (not the deprecated maven plugin) would produce the expected maven-metadata.xml). We're not reinventing the wheel here because we're using the aether library to publish, which keeps only one <snapshotVersion> per artifact. Maven should behave exactly the same because they rely on the same library.

... because we're using the aether library to publish, which keeps only one <snapshotVersion> per artifact. Maven should behave exactly the same because they rely on the same library.

isn't the aether library deprecated in favor of apache maven artifact resolver?

I don't understand why this thread is resurrected: the comments above clearly states that using maven-publish (not the deprecated maven plugin) would produce the expected maven-metadata.xml). We're not reinventing the wheel here because we're using the aether library to publish, which keeps only one <snapshotVersion> per artifact. Maven should behave exactly the same because they rely on the same library.

What I find disappointing is the fact that Gradle produces maven-metadata.xml files that contain only one version of the snapshot. If you have more than one build of that snapshot, the older ones will not be properly included and listed. In my opinion, (and I believe I know a thing or two about Maven metadata, as well as artifact repository managers), this is simply incorrect support. I flagged this three years ago and provided a really detailed explanation of the problem and how to solve it. In my opinion, this one snapshot build per maven-metadata.xml is incomplete support.

Being passive aggressive won't help you get a fix. Actually quite the opposite. This thread was closed _with an explanation_, saying we behave like Maven and at the time of the resolution, Gradle was using exactly the same aether library as Maven did so it was quite unlikely there would be a difference. In particular, Stefan showed real world examples of libraries published as such which show a single snapshotVersion entry. You didn't comment after this was closed, which, for us, means that you agree with the decision.

Now, if that's not the case:

  1. either we were wrong in the beginning, and it's a mistake, we can reopen an issue
  2. either the behaviour of Maven changed once again, and we need to support it (or not, we have choices and hard decisions to make)

In both cases creating a new issue would make sense since in the initial issue here, we were talking about the maven plugin which was deprecated in favor of maven-publish, and the latter behaves differently.

@melix Neither the maven plugin nor the maven-publish plugin behaves like Maven. If we should be opening a new issue about it, then we'll do that. @carlspring ?

Was this page helpful?
0 / 5 - 0 ratings

Related issues

bmuschko picture bmuschko  路  52Comments

eskatos picture eskatos  路  49Comments

bsideup picture bsideup  路  115Comments

JehandadK picture JehandadK  路  36Comments

lacasseio picture lacasseio  路  102Comments