Exist: [feature] enable appending to JAVA_TOOL_OPTIONS in docker-compose.yml

[eduarddrenth]

It would be very nice to be able to do something like:

    environment:
      - JAVA_TOOL_OPTIONS=${JAVA_TOOL_OPTIONS} -Dcaptchakey=${captchakey} -Dfrom=${from} -Dbcc=${bcc} -Dsmtp=${smtp}

in docker-compose.yml.

At present this is blocked because of distroless.

Personally I would consider supporting a non distroless Dockerfile with all advantages that will bring you, like variable substitution, reading logfiles other than stdout and having a shell to check things in the container..

[duncdrum]

So there are two things here, if you wish to add another docker image to the official publishing channel, I suggest creating a PR with your suggested changes and to join one of the community calls to discuss your changes and how the new images will be maintained with the core devs. As an open source project contributions are always welcome.

As for appending JAVA_TOOL_OPTION I think there needs to be a balance, between providing an optimized configuration, and letting ppl roll their own config. The image already provide some means to customize config settings, if one is missing we can add them. But it is very easy to kill your exist with a poor config, and I see little value in having to debug other ppls choices here, when they can roll their own image and deal with it themselves.

[eduarddrenth]

Happy to do a PR and to maintain a non distroless version

I'll see if I can join a call

[eduarddrenth]

where can I find info on community calls? Can't find it

[duncdrum]

https://github.com/eXist-db/exist#open-community-calls It’s all in the readme

[adamretter]

Also if you join the Slack channel - the community calls are announced
there too

On Sat, 18 Jul 2020 at 23:18, Eduard Drenth [e-mail]
wrote:

where can I find info on community calls? Can't find it

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/eXist-db/exist/issues/3484#issuecomment-660543923,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAJUTOI4XGKXRNSRGNWIWY3R4IGTDANCNFSM4O4QBK3A
.

--
Adam Retter

skype: adam.retter
tweet: adamretter
http://www.adamretter.org.uk

[eduarddrenth]

Links to code:

Docker stuff: https://bitbucket.org/fryske-akademy/online-dictionaries/src/master/docker/
Reading properties: https://bitbucket.org/fryske-akademy/online-dictionaries/src/master/src/main/java/org/fryske_akademy/onfw/properties/

The running app: https://web2.fa.knaw.nl/exist/apps/onfw/index.html?lang=en

[duncdrum]

as per today's community call we'll explore two basic option

• publish a second set of :debug images with an appropriate tag, that use :debug version of distroless in the final build stage, so users have access to the shell tools they are used to. These images should be documented and clearly marked as not recommended for production.
• implement a secure way for users to load property files in the existing images.

[eduarddrenth]

Working with basePath and docker secrets now...

load module with path to property fiiles in conf.xml:

            <module uri="http://exist-db.org/xquery/properties"
                    class="org.fryske_akademy.onfw.properties.PropertiesModule">
                <parameter name="basePath" value="/run/secrets"/>
            </module>

define a docker secret to properties:

once: docker secret create ${APPNAME}.properties ${VOLUMEROOT}/${APPNAME}/config/${APPNAME}.properties

use the docker secret in containers, i.e. docker-compose.yml:

    secrets:
      - teidict.properties
secrets:
  teidict.properties:
    external: true

load the properties in xql:

declare namespace properties="http://exist-db.org/xquery/properties";
declare variable $teidict:props := properties:loadProperties("teidict.properties");

[eduarddrenth]

With the secrets approach property changes will not be visible in the container. This may or may not be an advantage

[duncdrum]

Can you elaborate on that, it sounds desirable to me to be that way, but maybe I m missing something.

[eduarddrenth]

Often it is desirable that changing application properties do not require a restart.
Other properties should be unmodifiable and do require a restart.

If you 'mount' properties using docker secret the first use-case isn't supported, if you mount properties using docker volume it is.

My property solution supports docker volume, docker secret and non-docker.

In my java applications I often work with properties like this in CDI beans

@Inject @Property private int sheetWidth;

The configuration library I use takes care of looking up and initializing properties and changing values when the properties file is changed. This feature wouldn't be supported using docker secret to mount properties.

[duncdrum]

@eduarddrenth my main concern is that use-case via secrets I would expect a necessary restart there, so to me that's a plus.

As for the volume approach neat, we just need to document the concern, and notch folks in the right direction OOTB.

[eduarddrenth]

This is a bit old, in the mean time I published: https://search.maven.org/search?q=a:exist-db-addons

[eduarddrenth]

That was to early, wanted to type some more

[eduarddrenth]

In docker-compose.xml I have:

    secrets:
      - ${APPNAME}.properties

in conf.xml I have:

            <module uri="http://exist-db.org/xquery/properties"
                    class="org.fryske_akademy.exist.properties.PropertiesModule">
                <parameter name="basePath" value="/run/secrets"/>
            </module>

in the application I do:

declare variable $teidict:props := properties:loadProperties("teidict.properties");

declare function teidict:getProperty($key as xs:string) as xs:string {
    teidict:getProperty($key,"")
};

declare function teidict:getProperty($key as xs:string, $default as xs:string) as xs:string {
    if (map:contains($teidict:props,$key)) then
        map:get($teidict:props,$key)
    else
        $default
};

works like a charm, cooperates with docker secret, confined to basepath

[duncdrum]

Yes sorry, live got busy and then i was on holiday.
We should document that possibility, in the documentation repo. We could also consider to generate a default entry to the container's conf.xml via pom. Do you consider your app to be stable for inclusion in the core repo build?

[eduarddrenth]

absolutely stable yes. The same lib also holds a DataSyncTask that is very useful I think. From the javadoc:

/**
 * Sync files in a directory to a collection recursively. Files and collections that are new or newer than
 * the one in the target collection will be written to the collection. Files and collections that are
 * not present in the source directory will be removed from the collection, this can be turned off
 * via {@link DataSyncTask#REMOVE_FROM_COLLECTION_PARAM}.
 * Owner and group of collections and documnets can be given in parameters, otherwise they will be the same as
 * the owner and group of the parent collection of the {@link #COLLECTION_PARAM}.
 * After syncing cache is cleared to prevent problems, this can be turned off via {@link #CLEAR_CACHE_PARAM}.
 * NOTE that the sync will partially succeed when during syncing an exception occurs, collections and files added
 * or removed before the exception will remain added/removed.
 * Meant to be used as a start-up task, {@link DataSyncTaskCron} is meant to be scheduled as a cronjob.
 */

[eduarddrenth]

I'll do a pr, than it can be discussed