Jib: gradle jibDockerBuild fails with Could not expand ZIP message on second run

[infa-rbliznet]

Description of the issue:
I started to use jib for war files. My war file has read only files. When I run gradle jibDockerBuild second time it fail with message:
Execution failed for task ':bundle-svc:jibExplodedWar'.

Could not expand ZIP '.../bundle-svc/build/libs/bundle-service.war'.

Expected behavior: It shouldn't fail and should complete successfully

Steps to reproduce: In existing gradle project that produces a war file, change a file to be read only. Run gradle jibDockerBuild

Environment: Mac Os X, Gradle 4.10.3

jib-gradle-plugin Configuration:

jib {
  from.image = 'localhost:5000/repo/bundle-service:latest'
  allowInsecureRegistries = true
  to.image = 'repo/bundle-service-jib'
  // For demonstration only: this directory in the base image contains a Tomcat default
  // app (welcome page), so you may first want to delete this directory in the base image.
  container.appRoot = '/opt/tomcat/webapps'
}

Log output:

Additional Information:
Changing original file permission can't be a workaround to me since I use Perforce as a VCS which sets readonly flag for all files that weren't checked out.Those files are packed to war as is with the standard gradle war task.

[loosebazooka]

It looks like your build is generating this war as read only? We only package exploded wars, so if we cannot expand it, we cannot package it.

[infa-rbliznet]

Hi loosebazooka,
You can see that war file is writable.
```
$ ls -l build/libs/bundle-service.war
-rw-rw-rw- 1 rbliznet admin 38347751 May 19 12:02 build/libs/bundle-service.war

   However resource files inside the war file are readonly. So first time war is unpacked fine and see those file permissions:

    ```
    $ ls -l build/jib-exploded-war/WEB-INF/classes/
    total 104
    -r--r--r--  1 rbliznet  admin   8981 May 17 23:28 BundleErrorBundle.properties
    -r--r--r--  1 rbliznet  admin  19784 May 17 23:28 BundleErrorBundle_ja.properties
    -r--r--r--  1 rbliznet  admin    177 May 17 23:28 SearchCriteriaErrorBundle.properties
    -r--r--r--  1 rbliznet  admin    407 May 17 23:28 SearchCriteriaErrorBundle_ja.properties
    -r--r--r--  1 rbliznet  admin   4626 May 17 23:28 bundle-service-config.properties.sample
    drwxr-xr-x  3 rbliznet  admin     96 May 17 23:28 com
    drwxr-xr-x  3 rbliznet  admin     96 May 17 23:28 db
    -rw-r--r--  1 rbliznet  admin     85 May 17 23:28 microservice.version

If I remove the whole folder then I can re-run
rm -rf build/jib-exploded-war && gradle jibDockerBuild

[infa-rbliznet]

In order to confirm that it is readonly file inside was problem, I did a simple test:

    $ rm -rf build/jib-exploded-war
    $ ls -l src/main/resources/
    total 120
    -r--r--r--@ 1 rbliznet  admin   8981 Jun  2  2018 BundleErrorBundle.properties
    -r--r--r--  1 rbliznet  admin  19784 Aug 24  2018 BundleErrorBundle_ja.properties
    -r--r--r--  1 rbliznet  admin    177 May 11  2017 SearchCriteriaErrorBundle.properties
    -r--r--r--  1 rbliznet  admin    407 May 11  2017 SearchCriteriaErrorBundle_ja.properties
    -r--r--r--  1 rbliznet  admin   4626 Jun 26  2018 bundle-service-config.properties.sample
    -r--r--r--  1 rbliznet  admin   6470 Jul 18  2018 bundle-service.properties.sample
    drwxr-xr-x  3 rbliznet  admin     96 Feb 14  2017 com
    drwxr-xr-x  3 rbliznet  admin     96 Jul  5  2018 db
    -r--r--r--  1 rbliznet  admin   1220 Jan 23  2018 log4j.properties.sample
    $ chmod -R +w src/main/resources/
    $ gradle jibDockerBuild
    $ gradle jibDockerBuild

That worked just fine, previously 2nd build was failing.

[chanseokoh]

Maybe we can unzip the WAR into a temp directory.

[mcabezas]

I'll take a look on this. jib Team please confirm if it is possible.

[chanseokoh]

Yes, we can unzip this into a temp directory. Judging from the reported issue, it looks like we are always unzipping a WAR. I don't see any problem with a temp directory; it will just be a different directory than jib-exploded-war. I think using a temp directory is actually better, since users can remove files from a WAR during development, and since we are not cleaning the exploded directory, it is possible that unwanted files get containerized currently.

[rajovictor]

@chanseokoh I am thinking of using a timestamp based temp folder within build folder to unzip the war.
That would mean, I need to invoke the clean task with the new temp folder path after the execution of all the tasks.
What happens if the build fails after ExplodeWar task?

[chanseokoh]

@rajovictor well asked! I think we don't need to invoke the clean task on the temp folder. I've recently fixed #1091, and I just created a new temp directory that will be auto-cleaned. No timestamp required. Take a look at #2069.

(And sorry about not getting back to you about #2078. It'll take some more time.)

[chanseokoh]

Hmm... actually, I can see that using TempDirectoryProvider on the Gradle side may not be easy. TempDirectoryProvider should survive until after image building is complete in order to retain the temporary directory it creates long enough. Not sure if we can make use of TempDirectoryProvider on Gradle.

[rajovictor]

Ok, I will give it a try with TempDirectoryProvider on Gradle.

[rajovictor]

OTOH, Why can't we delete the jib-exploded-war if it already exists and unzip again?

[chanseokoh]

That's a good point, but I think we may not be able to delete the directory in this case. The issue is that we cannot unzip it because there are some files or directories that cannot be deleted/overwritten.

[chanseokoh]

If TempDirectoryProvider doesn't play nicely with ExplodeWar task, we can think about abandoning ExplodeWar completely and do the exact same thing in GradleProjectProperties as in #2069. We will have to see.

[rajovictor]

I am using only ExplodedWarTask#setWarFile along with TempDirectoryProvider as you did in #2069 and it works.
I will modify the unit and integration tests and create a PR.

[rajovictor]

Created PR #2101

[chanseokoh]

Fixed by #2101.

[chanseokoh]

@infa-rbliznet we just released Jib 1.8.0, and I believe it fixes this issue.

@rajovictor thanks for your contribution to fix this!