Aws-sdk-java: com.amazonaws/aws-java-sdk-core: Transitive dependency conflict with `commons-logging`

[harold]

com.amazonaws/aws-java-sdk-core depends on both:
1) commons-logging "1.1.3"
2) org.apache.httpcomponents/httpclient "4.5.2" :arrow_right: commons-logging "1.2"

AKAICT com.amazonaws/aws-java-sdk-core switches out commons-logging to an older version out from under org.apache.httpcomponents/httpclient.

I haven't looked into what changed in commons-logging between "1.1.3" and "1.2", but there are code paths here that blow up (some incompatible access of inner types?!).

Perhaps the dependency on commons-logging can be removed from this lib entirely? Or at least be brought into transitive compatibility with org.apache.httpcomponents/httpclient?

Thanks in advance, and as ever, for your time and consideration.

[millems]

Thanks for your detailed report! Do you have any specific instances of issues you've encountered due to this conflict?

[harold]

I don't; none of our code calls any of this stuff directly. com.amazonaws/aws-java-sdk-core is a dep of a dep of a dep of a silly plugin we use for private jar repo management.

The problem I'm actually trying to solve showed up in our CI process, which runs on cloud machines that are not under my control and have some combination of various random JDKs, .m2 state, and other confounding config that make reproducing these errors tough. Some dep of a dep throws java.lang.NoSuchMethodError because it relied on an implementation detail of some lib it uses that got an old (or new) version swapped out from under it.

This particular little red arrow may or may not be the cause of my pain today, but I thought I'd flag it up here. Hope that helps.

[millems]

Hah! Thank you very much for taking the time!

[gordonsyme]

Hi there,

I'm hitting this issue too, we use enforcement mechanisms (Maven Enforcer for Java, :pedantic? :abort for Clojure) to make sure that we have no ambiguous dependencies in our dependency graph.
These tools force us to make an explicit decision about which version to use along with recording that decision.

Having a library we depend on with its own internal dependency conflicts requires quite a bit more work to clean up the dependency tree.

[debora-ito]

Hello! This is just a quick update letting you know that the SDK team has reviewed the feature request list for V1 and this one looks like a great candidate for a community PR, which we’ll help merge in and support.

[harold]

It looks like the dep is expressed here: https://github.com/aws/aws-sdk-java/blob/master/aws-java-sdk-core/pom.xml#L19

Which references this property: https://github.com/aws/aws-sdk-java/blob/master/pom.xml#L238

Moreover, It looks like the dep to httpclient has been upgraded to 4.5.5 in the two years since I reported this issue (https://github.com/aws/aws-sdk-java/blob/master/pom.xml#L257), but it also appears that httpclient 4.5.5 still depends on commons-logging 1.2 (https://mvnrepository.com/artifact/org.apache.httpcomponents/httpclient/4.5.5)

The two choices seem to be:
1) remove the explicit dependency that aws-java-sdk-core has on commons-logging; commons-logging is still available transitively, and then cannot be out of sync.
2) change the version of the commons-logging dep to 1.2; they at least match, but that property is shared among the various libs built from this repo, and so testing will be needed to ensure no breakage in other libs.

Someone who knows more about the testing infrastructure around this code, as well as the use of commons-logging in aws-java-sdk-core, should make the call.