Velero: Add --plugins flag to `velero install` to allow plugins to be included at install time

[skriss]

velero install should allow 1+ plugins to be installed during the main install process. This would allow out-of-tree cloud providers and other plugins to be installed seamlessly, without requiring a restart.

[skriss]

xref #1537

[nrb]

IMO, the values for --plugin should be container image URLs that would be passed into velero plugin add today.

[tlvenn]

This would be great as it would open the path to support this with the helm chart and to have a declarative / terraform compatible way of managing those plugins.

[tlvenn]

But actually it's 2 distinct paths given the helm chart is not invoking the cli to install velero... I should open a separate issue to track this.

[carlisia]

An issue for the Helm change was created: https://github.com/heptio/velero/issues/1819

[nrb]

Per discussion with the team, we're going to, for the time being, make installation a 2 step process - running velero install to generate the necessary objects in cluster, and velero plugin add to add any relevant object or snapshot plugins.

This is mostly temporary to get things split up per #1870. Longer term, we want to revisit how plugins are handled holistically so we have a clearer answer for how things like secrets may be handled, and the possibility of making plugins run as sidecars instead of init containers.

[skriss]

What's the reasoning for not adding the flag to velero install? IMHO it's simpler for the user to include it as part of the install command, and since it's required for velero to be functional, would be better to make it an atomic part of the install.

I'm fine revisiting things down the road, but I'm failing to see how making it a separate step right now is better than this.

[nrb]

What's the reasoning for not adding the flag to velero install?

This came up as confusing during one of our discussions on the implementations. A 2 step process was proposed, one for getting the resources into the cluster and one for installing the plugins. That's possible today.

I'm fine with adding the --plugins flag if the rest of the team is.

[carlisia]

It's fine by me.

[skriss]

To be clear, I definitely think there's room for improvement here in the install & setup flow, the relationships between plugin images, names, and BSLs/VSLs, what we validate at server startup, etc. that we should take a holistic look at - but I propose doing that separately.