Skaffold: Kustomize integration enhancements

[dlorenc]

Kustomize supports overlay-based "variants" to manage configuration across environments.

We could help support this in Skaffold in a few ways:

• Opionated "variant" naming and directory structure. Something like a "-dev" prefix or suffix on the yaml would be used during skaffold dev, but not run.
• Supporting multiple kustomization paths. skaffold dev can use one, skaffold run can use the other.

[dlorenc]

Please list any other ideas you have here!

[bhack]

How this will interact with Skaffold profiles?

[geekflyer]

I just "attempted" to use the existing skaffold kustomize integration but in its current state I think it's effectively unusable. One can basically cannot configure anything, not even the directory in which the kustomization.yaml lies can be configured (the code uses the hardcoded default path ./ which cannot be overriden: https://github.com/GoogleContainerTools/skaffold/blob/master/pkg/skaffold/deploy/kustomize.go#L53).
As a result also skaffold profiles don't help at the moment, because even with profiles I cannot configure the (non-configurable) kustomize integration.

[r2d4]

We can pretty easily make that configurable @geekflyer. Is there anything else that would be needed to unblock you?

[geekflyer]

@r2d4 That would be awesome! I think configuring the kustomization path is all I'd need for now. Thanks.

[tomster]

@r2d4 for me dev/staging/production "live" both in separate kustomize layers as well as their respective kubectl contexts. my current workflow is to switch the kubectl context and then run kustomize with the appropriate target (using skaffold only for local developement, bypassing kustomize)

the convention is that the base layer of kustomize is effectively the "devel" overlay and thus i can let skaffold simply run against that directory (manifests: - base/*.yaml).

however, skaffold has now spoiled me and i would like to use it against the staging and production overlays, as well :-)

perhaps simply allowing to override the kustomize directory on the command line would solve all of the above, so during development i would run skaffold dev as per usual and to apply it against staging i would do something like skaffold run overlays/staging

does that make sense?

[tomster]

of course, just after posting the above comment i discovered https://github.com/GoogleContainerTools/skaffold/pull/749 but then ran into https://github.com/GoogleContainerTools/skaffold/issues/788...

but still, i think being able to override the kustomizePath setting on the command line would be the preferred option for me.

[balopat]

With the current, flexible kustomize support all these things can be expressed via profiles. I'm hesitant to introduce the name of the overlay to be the same as the 'command' by default as people might use skaffold run for multiple profiles - maybe overlays/profile_name could be interesting - but what if there are multiple profiles activated? I'm going to close this for now unless somebody feels strongly about introducing this convention.