Jib: ExtraDirectory should allow for specifying a target

[alkoclick]

Description of the issue:
The extraDirectory object is fairly simple to use but imposes an important constraint (mimicking your intended folder structure under the source). This, in turn, makes it essential to use other plugins/functionality to configure the intended directory structure. This code delays execution, introduces duplication (as you will probably already have the desired artifacts, but not in the desired configuration) and blobs the code with what could be a very pretty one-liner.

Expected behavior:
extraDirectories {
paths + targets: ...
permissions: ...
}

Steps to reproduce:
N/A

Environment:
All

Additional Information:
I may be able to help out on this PR?

Thanks for building an awesome tool!

[chanseokoh]

Hi @alkoclick,

Thanks for the feedback. Adding targets (which effectively defaults to / now) somewhere under extraDirectories sounds good to me. Combined with #1020, I think it'll make life easier for many people. It'd be nice if you could help us out, but before that, we need to have an internal discussion about if we really want to do this.

[scripnichenko]

Feature required for my case as well!

[slowr]

Any update on this @chanseokoh ?

My use case for this feature request is that I want to also deploy some libraries and configurations for tomcat when copying the war file but also some scripts.

For example:

[{from: `custom-conf/server.xml`, to: `usr/local/tomcat/conf/server.xml`},
{from: `custom-scripts/*.sh`, to: `usr/local/bin/`}]

I can copy them to the base image beforehand but the problem is that I want to be able to update and change them easily instead of recreating the whole base image. I would love to try and create a PR on this if you think this feature is good to have.

[chanseokoh]

Hi @slowr,

We really appreciate external contributions. We do think this feature is very useful and hoped to get to work on this at some point. Sounds like you are serious about creating a PR, so we will have a quick internal design discussion this week to decide how the configuration in Gradle and Maven should look like.

I can copy them to the base image beforehand

If you haven't noticed, the default "to" directory in the image is always /, so for now you will have to place your files in the following way.

<project root>/src/main/jib/usr/local/tomcat/conf/server.xml
<project root>/src/main/jib/usr/local/bin/*.sh

Note here that the default "from" (jib.extraDirectories) directory is <project root>/src/main/jib and is configurable. So if you want, you can go with

<project root>/src/main/custom-conf/usr/local/tomcat/conf/server.xml
...

and set jib.extraDirectories.paths = ['src/main/custom-conf', 'src/main/custom-scripts', ...].

[slowr]

Thanks a lot for answering!

If you haven't noticed, the default "to" directory in the image is always /, so for now you will have to place your files in the following way.

<project root>/src/main/jib/usr/local/tomcat/conf/server.xml
<project root>/src/main/jib/usr/local/bin/*.sh

I know about this solution but the problem is that I have multiple projects that share the same configuration files so I can't actually put these files inside each project. If I am missing something that can make this work for multi-project setup please tell me!

[chanseokoh]

@slowr someone was having the same problem as yours, and I suggested this workaround. I didn't get a response, so not sure if it worked for them. But I think this will allow you to put the same config files in a single place and share them in a multi-module setup.

[chanseokoh]

And come to think of it, this issue is about allowing to set the "to" directory, so you will still have the same config-sharing problem even this is fixed. (Just wanted to make sure that you clearly understood what this issue is about.)

[slowr]

And come to think of it, this issue is about allowing to set the "to" directory, so you will still have the same config-sharing problem even this is fixed. (Just wanted to make sure that you clearly understood what this issue is about.)

Yes that is true, it didn't click me before! I think getting the work-around and adding files from other projects is doable but having to replicate the absolute path with directories seems to be a bit hacky.

[chanseokoh]

Anyways, do you still feel like you want to put up a PR? If so, what we decided is

extraDirectories.paths = [
  {
    from: ..., // required
    into: ...  // "into" is optional and can be omitted ('/' by default)
  }, {from: ..., into: ...}, ...
]

while at the same retaining the backward-compatibility that the user can continue to do

extraDirectories.paths = [ 'some/src/dir', 'another/dir', ... ]

[slowr]

I found a solution to my issue by creating a task that copies everything into a temp directory with the structure you already support and then adding that directory as an extraDirectory. Something like this:

tasks.register('jib-directories') {
    doLast {
        String jibDir = "$project.buildDir/jib-directories", tomcatDir = "/usr/local/tomcat", deployDir = "/usr/local/data"

        // copy tomcat files
        copy {
            from rootProject.files('config/tomcat-modifications/conf')
            into "$jibDir/$tomcatDir/conf"
        }
        // copy deployment files
        ['bin', 'lib', 'systems', 'xml', 'etc'].each { String dest ->
            copy {
                from project(':deployment').files("$dest")
                into("$jibDir/$deployDir/$dest")
            }
        }
    }
}

jib {
    ...
    extraDirectories.paths = ['.build/jib-directories']
}

tasks.named('jib').configure {
    dependsOn jib-directories
}

So, do you think this solution is better than introducing into? I will take a look to create a PR if you think that into is a better approach.

[chanseokoh]

@slowr thanks for posting the tricks. This will help other users.

If you think you are good with your solution and don't have (or just don't want to put any) time on introducing into, that's totally OK. We'll leave this issue open, and hopefully, we will get to this at some point ourselves. To be honest, we anticipate implementing this feature while taking care of all the corner cases would not be an easy task and take up a lot of your time. The code review could be dragged for a pretty long time. In any case, I really appreciate your previous contribution about the Gradle tasks!

[slowr]

if it's not get picked up and I have more time I will take a look and make a PR for this issue so we can have a more clear way to copy files in the container.

Thanks a lot for this great project!

[chanseokoh]

@alkoclick @scripnichenko @slowr @padyx @celcius112 @student-06 @yoshi10321

Jib 2.3.0 is released with this very handy feature. Now you can designate target directories for <extraDirectories> / jib.extraDirectories.

  extraDirectories {
    paths {
      path {
        // copies the contents of 'src/main/extra-dir' into '/' on the container
        from = file('src/main/extra-dir')
      }
      path {
        // copies the contents of 'src/main/another/dir' into '/extras' on the container
        from = file('src/main/another/dir')
        into = '/extras'
      }
    }
  }

<configuration>
  <!-- Copies files from 'src/main/custom-extra-dir' and '/home/user/jib-extras' instead of 'src/main/jib' -->
  <extraDirectories>
    <paths>
      <!-- Copies from 'src/main/custom-extra-dir' into '/' on the container. -->
      <path>src/main/custom-extra-dir</path>
      <!-- Copies from '/home/user/jib-extras' into '/extras' on the container -->
      <path>
        <from>/home/user/jib-extras</from>
        <into>/extras</into>
      </path>
    </paths>
  </extraDirectories>
</configuration>