Elasticsearch: Add update parameter to bin/plugin, for easier updating of plugins

[jakommo]

With the frequent release cycle (and the need to update every elastic plugin), it would be nice to add an update parameter to bin/plugin, that does bin/plugin remove ... and bin/plugin install ... in one step.

[uboness]

Personally I'm -1 on this one... I really want to discourage ppl from blindly updating plugins and hoping everything "just works". I much rather force ppl to read upgrade docs that will take them step by step through the upgrade process.

[ppf2]

+1 from additional users to have an update plugin option.

Also +1 on @uboness 's comment on discouraging people from blindly updating plugins. Perhaps we can build some intelligence into the update plugin script. For example, in addition to verifying that the plugin version is compatible with the ES version (a given), perhaps we can have a way to tag a new plugin version with a "breaking" flag (for plugin updates that have its own breaking changes), and when the user attempts to update to this plugin, we can prompt them to acknowledge it (with a link to the breaking change documentation for the plugin) before allowing the actual update to run. But this also means that we will need to start creating a breaking changes section in our documentation for each plugin, etc.. just a thought ...

[dadoonet]

@ppf2 FYI we do have such doc in breaking changes. For example in 3.0: https://www.elastic.co/guide/en/elasticsearch/reference/master/breaking_30_plugins.html

[dadoonet]

Also IMO we should not have breaking changes between minor versions (second digit in X.Y.Z).
It means that we should always deprecate but keep the compatibility with old settings, old file formats.
Like we try to do in elasticsearch itself.

For sure, updating from 2.x to 3.x should not be allowed in general.

[clintongormley]

I'm -1 on this as well. Besides the reasons stated by @uboness above, we also have to consider that plugins come from the community as well, which can come from github, maven, and http URL, wherever. Plugin locations may change, be renamed, be discontinued etc. While this seems like a simple change to implement, it comes with a boatload of complexity.

[clintongormley]

We're not going to add the update option, but we should document the need to upgrade plugins during the rolling upgrade process. Closing in favour of https://github.com/elastic/elasticsearch/issues/15389

[ludwig-gramberg]

so this is what happens in real life:

1. updating server with ES on a regular basis
2. ES is just one package of many so it updates
3. sometime later server needs a reboot
4. server reboots, ES doesnt start because plugins are now incompatible with ES version
5. monitoring goes bananas and we rush to solve the issue manually

so when i do have a package in linux which is managed/updated by a package manager it should do so entirely and leave the application in a rebootable state.

so how about a practical solution? of course major version upgrades require some reading but why does it require manual work to upgrade from 5.5.0 to 5.5.1?

[jakommo]

@ludwig-gramberg I think this is not related to this issue. This issue was about an update param of the plugin manager, but in the case you are describing you would still need to issue that plugin update manually.

I highly recommend using apt hold or yum lock with the elasticsearch package, to not update it at random times. There are way worse things that could happen if you upgrade parts of your cluster without supervision.

so when i do have a package in linux which is managed/updated by a package manager it should do so entirely and leave the application in a rebootable state.

Those plugins are not part of the deb/rpm package and it could even be third party plugins.
I don't think that automatically updating them would guarantee that the node comes back after a reboot.

[ludwig-gramberg]

@jakommo and having not come up guaranteed is better then it maybe not coming up? but yeah, guess i will go with the lock option... and then i will just ignore most updates, and maybe every once in a while i might update... hope nothing security related ever comes up...

[jakommo]

@jakommo and having not come up guaranteed is better then it maybe not coming up?

I would prefer it not coming up more than coming up and joining the cluster in a potential weird state that could even cause issues to the rest of the cluster.
Anyway, what I'm trying to say is that I would not recommend doing none supervised upgrades of any application that is mission critical, in general.

and then i will just ignore most updates, and maybe every once in a while i might update... hope nothing security related ever comes up

I don't see why you should stop upgrading because of this.
You can subscribe to https://discuss.elastic.co/c/security-announcements.rss for security announcements and then decide if you feel it's worth upgrading.