Brigade: Enhancement: Add worker command flag to the controller

[krancour]

Using brig project create, I can override the worker image (to use something custom) at the project level and this includes the option to override the command that is invoked in the worker container.

Supposing, however, that I'd like Brigade to default to using my custom worker image, I can do so by setting the right values during helm install, which will, in turn, set the right flags on the controller to affect what image and pull policy are used, _however_, unlike when using brig project create, I have no option for specifying the command that should be invoked in the worker container.

To correct this, I'd like to propose doing the following adding a new controller flag for specifying the worker command and also exposing that option through the Helm chart.

This has some minor ripples that are easily accounted for, but cannot be ignored:

Currently, when creating a new project via brig project create, the project-level _default_ worker command (i.e. if you just hit enter) is yarn -s start. This may not be the right command for whatever custom worker image is in use. I'd like to propose changing this default to the empty string. As it turns out, this would work just fine, because this is how we _already_ select the command to use when creating the worker pod:

cmd := []string{"yarn", "-s", "start"}
if cmdBytes, ok := project.Data["workerCommand"]; ok && len(cmdBytes) > 0 {
  cmd = strings.Split(string(cmdBytes), " ")
}

i.e. yarn -s start is _already_ hard-coded as a universal default, so it doesn't _also_ need to be the default value for the project-level worker command.

After adding the new flag, this logic would change slightly to be like this:

cmd := []string{"yarn", "-s", "start"}
if config.WorkerCommand != "" {
  cmd = strings.Split(config.WorkerCommand, " ")
}
if cmdBytes, ok := project.Data["workerCommand"]; ok && len(cmdBytes) > 0 {
  cmd = strings.Split(string(cmdBytes), " ")
}

Effectively, the code above starts with the default and then progressively applies global and project-level overrides, if applicable.

The last thing to consider is what will happen if an older version of brig is used with a newer version of the controller, or vice versa.

Older brig with newer controller will still have the project-level worker command default to yarn -s start just as it does today. The newer controller will honor this, same as it does today, so it's effectively no change from the status quo.

Newer brig with older controller will use the empty string at the project-level worker command default and the older controller is already tolerant of that, so again, the status quo is maintained.

In the ideal case, however, where one is using _both_ newer brig _and_ a newer controller, a global override of the worker command would finally be possible.

[krancour]

I should add that I'd be happy to implement this myself if maintainers agree with this proposal.

[radu-matei]

Yeah, once we consider Brigade controllers that don't necessarily extend on the default worker that we ship by default, this becomes a requirement, and in my opinion, as long as the defaults still work when upgrading brig and / or the chart, we should definitely have this.