Aws-sdk-android: Conflicting compile dependencies in 2.4.0

[amake]

Three dependencies of the aws-android-sdk-s3 package changed from being test scope in 2.3.9 to being the default compile scope in 2.4.0. These dependencies include both org.apache.commons:commons-io:1.3.2 and commons-io:commons-io:2.4, which conflict in that they both provide the class org.apache.commons.io.CopyUtils. This causes Android apps depending on the com.amazonaws:aws-android-sdk-s3 package to fail to build.

There are two specific paths to failure I've confirmed:

1. Not using proguard minify: the :app:transformClassesWithDexForRelease task will fail
2. Using proguard minify and keeping dependency classes un-obfuscated (which is generally good practice): the :app:transformClassesAndResourcesWithProguardForRelease task will fail

Is it possible that moving the dependencies from test to compile scope was a mistake? I would also question the necessity of including both versions of commons-io, but perhaps there is a good reason for that.

• What service are you using?
  S3 (com.amazonaws:aws-android-sdk-s3)
• In what version of SDK are you facing the problem?
  2.4.0
• Is the issue limited to Simulators / Actual Devices?
  N/A
• Can your problem be resolved if you bump to a higher version of SDK?
  No
• Is this problem related to specific Android/OS version?
  No
• Can you give us steps to reproduce with a minimal, complete, and verifiable example? Please include any specific network conditions that might be required to reproduce the problem.
• Please include a stacktrace if applicable.

1. Create a new, empty project with the New Project wizard in Android Studio 2.3
2. Add the dependency compile 'com.amazonaws:aws-android-sdk-s3:2.+' to app/build.gradle
3. Run the assembleRelease task (e.g. ./gradlew assembleRelease on the command line)

Resulting Gradle log (excerpt):

:app:transformClassesWithDexForRelease FAILED

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':app:transformClassesWithDexForRelease'.
> com.android.build.api.transform.TransformException: com.android.ide.common.process.ProcessException: java.util.concurrent.ExecutionException: com.android.dex.DexException: Multiple dex files define Lorg/apache/commons/io/CopyUtils;

Gradle log excerpt for the proguard minify scenario (included for searchability):

  Copying resources from program jar [/Users/amake/.m2/repository/commons-io/commons-io/2.4/commons-io-2.4.jar] (filtered)
Warning: Exception while processing task java.io.IOException: Can't write [/Users/amake/Code/myapp/build/intermediates/transforms/proguard/release/jars/3/1f/main.jar] (Can't read [/Users/amake/.m2/repository/commons-io/commons-io/2.4/commons-io-2.4.jar(;;;;;;**.class)] (Duplicate zip entry [commons-io-2.4.jar:org/apache/commons/io/CopyUtils.class]))
:app:transformClassesAndResourcesWithProguardForRelease FAILED

[karthiksaligrama]

we are aware of this issue, it will be fixed in the next release. For now can you please change the gradle dependency to

compile('com.amazonaws:aws-android-sdk-s3:2.+'){
    exclude module: 'commons-io'
}

[amake]

Thanks for confirmation. When will the next release be?

And in case anyone is confused: the exclude line needs to be exclude module: 'commons-io'.

[karthiksaligrama]

thanks updated comment. Next release is being tracked in milestone 2.4.1. You should see updates there.

[snkashis]

I set up my excludes as such,

    compile ('com.amazonaws:aws-android-sdk-core:2.4.0'){
        exclude module: 'commons-io'
    }
    compile ('com.amazonaws:aws-android-sdk-s3:2.4.0'){
        exclude module: 'commons-io'
    } 

and also as

    compile 'com.amazonaws:aws-android-sdk-core:2.4.0'
    compile ('com.amazonaws:aws-android-sdk-s3:2.4.0'){
        exclude module: 'commons-io'
    }

but these lines below still block my release build. @karthiksaligrama related?

Warning:org.bouncycastle.jce.provider.X509LDAPCertStoreSpi: can't find referenced class javax.naming.NamingEnumeration
Warning:org.bouncycastle.jce.provider.X509LDAPCertStoreSpi: can't find referenced class javax.naming.NamingException
Warning:org.bouncycastle.jce.provider.X509LDAPCertStoreSpi: can't find referenced class javax.naming.directory.Attribute
Warning:org.bouncycastle.jce.provider.X509LDAPCertStoreSpi: can't find referenced class javax.naming.directory.Attributes
Warning:org.bouncycastle.jce.provider.X509LDAPCertStoreSpi: can't find referenced class javax.naming.directory.DirContext
Warning:org.bouncycastle.jce.provider.X509LDAPCertStoreSpi: can't find referenced class javax.naming.directory.InitialDirContext
Warning:org.bouncycastle.jce.provider.X509LDAPCertStoreSpi: can't find referenced class javax.naming.directory.SearchControls
Warning:org.bouncycastle.jce.provider.X509LDAPCertStoreSpi: can't find referenced class javax.naming.directory.SearchResult
Warning:org.bouncycastle.jce.provider.X509LDAPCertStoreSpi: can't find referenced class javax.naming.directory.InitialDirContext
Warning:org.bouncycastle.jce.provider.X509LDAPCertStoreSpi: can't find referenced class javax.naming.directory.SearchControls
Warning:org.bouncycastle.jce.provider.X509LDAPCertStoreSpi: can't find referenced class javax.naming.directory.SearchResult
Warning:org.bouncycastle.jce.provider.X509LDAPCertStoreSpi: can't find referenced class javax.naming.NamingEnumeration
Warning:org.bouncycastle.jce.provider.X509LDAPCertStoreSpi: can't find referenced class javax.naming.directory.Attribute
Warning:org.bouncycastle.jce.provider.X509LDAPCertStoreSpi: can't find referenced class javax.naming.directory.Attributes
Warning:org.bouncycastle.jce.provider.X509LDAPCertStoreSpi: can't find referenced class javax.naming.directory.DirContext
Warning:org.bouncycastle.util.AllTests: can't find referenced class junit.textui.TestRunner
Warning:org.bouncycastle.x509.util.LDAPStoreHelper: can't find referenced class javax.naming.NamingEnumeration
Warning:org.bouncycastle.x509.util.LDAPStoreHelper: can't find referenced class javax.naming.NamingException
Warning:org.bouncycastle.x509.util.LDAPStoreHelper: can't find referenced class javax.naming.directory.Attribute
Warning:org.bouncycastle.x509.util.LDAPStoreHelper: can't find referenced class javax.naming.directory.Attributes
Warning:org.bouncycastle.x509.util.LDAPStoreHelper: can't find referenced class javax.naming.directory.DirContext
Warning:org.bouncycastle.x509.util.LDAPStoreHelper: can't find referenced class javax.naming.directory.InitialDirContext
Warning:org.bouncycastle.x509.util.LDAPStoreHelper: can't find referenced class javax.naming.directory.SearchControls
Warning:org.bouncycastle.x509.util.LDAPStoreHelper: can't find referenced class javax.naming.directory.SearchResult
Warning:org.bouncycastle.x509.util.LDAPStoreHelper: can't find referenced class javax.naming.directory.InitialDirContext
Warning:org.bouncycastle.x509.util.LDAPStoreHelper: can't find referenced class javax.naming.directory.SearchControls
Warning:org.bouncycastle.x509.util.LDAPStoreHelper: can't find referenced class javax.naming.directory.SearchResult
Warning:org.bouncycastle.x509.util.LDAPStoreHelper: can't find referenced class javax.naming.NamingEnumeration
Warning:org.bouncycastle.x509.util.LDAPStoreHelper: can't find referenced class javax.naming.directory.Attribute
Warning:org.bouncycastle.x509.util.LDAPStoreHelper: can't find referenced class javax.naming.directory.Attributes
Warning:org.bouncycastle.x509.util.LDAPStoreHelper: can't find referenced class javax.naming.directory.DirContext
Warning:there were 46 unresolved references to classes or interfaces.
Warning:Exception while processing task java.io.IOException: Please correct the above warnings first.

[432player]

This helped me a lot!
In fact I couldn't find an answer to my problem anywhere on StackOverflow!
Only here on this discussion I have found the answer.

compile('com.amazonaws:aws-android-sdk-s3:2.+'){
exclude module: 'commons-io'
}

[iam1492]

@amake To resolve this issue, I add following code to my gradle file.
configurations {
all*.exclude group: 'org.apache.commons'
}

@snkashis And I open new issue #276 about proguard error.

[yesidlazaro]

compile ('com.amazonaws:aws-android-sdk-core:2.4.0'){
exclude module: 'commons-io'
}
compile ('com.amazonaws:aws-android-sdk-s3:2.4.0'){
exclude module: 'commons-io'
}

this works for me.

[karthiksaligrama]

fixed with 2.4.1

[hvar90]

with version 2.4.1 i have this error yet

Error:Execution failed for task ':app:transformClassesWithJarMergingForRelease'.
com.android.build.api.transform.TransformException: java.util.zip.ZipException: duplicate entry: org/apache/commons/io/CopyUtils.class

[iam1492]

+1 to @hvar90
@karthiksaligrama it seems not fixed yet.

[amake]

@karthiksaligrama The issue is not really fixed. You restored commons-io:commons-io:2.4 to the test scope but not org.apache.commons:commons-io:1.3.2 (diff) so anyone legitimately using a modern commons-io is still unable to build.

Unless something has changed in 2.4.1 that requires org.apache.commons:commons-io:1.3.2 in the compile scope, it should be changed back to test scope as well.