Gocd: No way to override bundled plugin when using Docker

[chriswhelix]

Issue Type

Bug Report

Summary

The latest GoCD server version (17.11.0) does not come bundled with the latest YAML config plugin version (0.6.0). When using the official GoCD Docker image to run the server, there is no way to override the bundled plugin.

Environment

Using a Dockerfile that looks like:

FROM gocd/gocd-server:v17.11.0

ENV GOCD_YAML_CONFIG_VERSION=0.6.0

RUN mkdir -p /godata/plugins/external && cd /godata/plugins/external \
    && wget "https://github.com/tomzo/gocd-yaml-config-plugin/releases/download/${GOCD_YAML_CONFIG_VERSION}/yaml-config-plugin-${GOCD_YAML_CONFIG_VERSION}.jar"

Desired Results

GoCD uses plugin version 0.6.0.

Actual Results

GoCD uses bundled plugin version 0.5.0. Note, this is as per documentation; the behavior is expected, but deeply frustrating.

On a persistent server, I could remove the bundled plugin, or install the new version over it, then restart GoCD. But because we're using Docker to manage plugin installation, all plugin installation activities occur before the first run of the server. So whatever I might do in the Dockerfile, as soon as the server boots up, it writes out its bundled plugins, overriding my desired versions.

Possible Fix

My immediate problem would be solved by bundling the latest version of the plugin (CC/ @tomzo).

However, it also seems to me there should be some way, as a user of GoCD via Docker, to disable or replace bundled plugins. I would view it as a defect that, in the context of Docker, certain plugins have been made mandatory and can not be upgraded or disabled (unless I'm missing something).

[bdpiprava]

@chriswhelix --- You are right about the bundled plugin behaviour. On server boot up it first loads the bundled plugins and then the external plugin. Only way I can see is,

1. Start gocd/gocd-server:v17.11.0 with volume mount.

docker run --name go-server -d -p8153:8153 -p8154:8154 -v <PATH_TO_DATA_DIR>:/godata gocd/gocd-server:v17.11.0

1. Replace bundled plugin under /plugins/bundled

wget -O gocd-yaml-config-plugin.jar  https://github.com/tomzo/gocd-yaml-config-plugin/releases/download/0.6.0/yaml-config-plugin-0.6.0.jar

1. Restart go server

docker restart go-server

[tomzo]

I recall @arvindsv talking on gitter that this is legacy behavior that should be removed. It used to be that bundled plugins had move quality testing and hence their priority when loading.
I think that currently we should simply switch to preferring the latest version, regardless if it is in /bundled or /external folder.

[ketan]

I think that currently we should simply switch to preferring the latest version, regardless if it is in /bundled or /external folder.

I agree with this argument. I'll defer to @arvindsv for the final vote to confirm this behavior.

@bdpiparva just pointed me to this method that seems to be in the way, and it the current behavior (that bundled overrides external) seems by design. https://github.com/bdpiparva/gocd/blob/fcbfd6443f6e9edc24df7fb49ad651a78bc9dac3/plugin-infra/go-plugin-infra/src/com/thoughtworks/go/plugin/infra/listeners/DefaultPluginJarChangeListener.java#L120-L124. According to @bdpiparva, looking at the old "cruise" code does not provide any historical context either.

[arvindsv]

Yes, this needs to change. Currently, it's too dependent on the filesystem and the order the directories are considered. How about we use these principles?

1. (ID + version) becomes important.
2. Distinction between bundled and external becomes unimportant.
3. Upon startup all plugins, across bundled and external are considered and loaded. Only the latest version is loaded if ID is duplicated. Irrespective of where it is.
4. API to load plugins and override running plugins (to upgrade version, for example).
5. Hacky workaround to load plugins automatically during development goes away.

[stale[bot]]

This issue has been automatically marked as stale because it has not had activity in the last 90 days.
If you can still reproduce this error on the master branch using local development environment or on the latest GoCD Release, please reply with all of the information you have about it in order to keep the issue open.
Thank you for all your contributions.