Skaffold: [proposal] Support profiles containing only diffs without redefining the whole section

[hypnoglow]

Currently, when using profiles, we have to specify the full build or deploy section, even if profiles differ in one field.

If we could specify only differences in profiles, the skaffold.yaml file would look much less clunky and with almost no duplications.

An example of how the skaffold.yaml file looks like currently when you want to use profiles that differ in one field:

apiVersion: skaffold/v1alpha2
kind: Config
build:
  artifacts:
  - imageName: my/image
profiles:
- name: dev
  deploy:
    helm:
      releases:
      - name: my-release
        namespace: my-namespace
        chartPath: deployment/my-chart
        valuesFilePath: deployment/environments/dev/chart-values.yaml
        values:
          "image.repository": my/image
        setValues:
          "image.tag": ""
- name: staging
  deploy:
    helm:
      releases:
      - name: my-release
        namespace: my-namespace
        chartPath: deployment/my-chart
        valuesFilePath: deployment/environments/staging/chart-values.yaml
        values:
          "image.repository": my/image
        setValues:
          "image.tag": ""

And this is an example how the skaffold.yaml file could look like with only diffs in profiles section:

apiVersion: skaffold/v1alpha2
kind: Config
build:
  artifacts:
  - imageName: my/image
deploy:
  helm:
    releases:
    - name: my-release
      namespace: my-namespace
      chartPath: deployment/my-chart
      values:
        "image.repository": my/image
      setValues:
        "image.tag": ""
profiles:
- name: dev
  deploy:
    helm:
      releases:
        my-release:
          valuesFilePath: deployment/environments/dev/chart-values.yaml
- name: staging
  deploy:
    helm:
      releases:
        my-release:
          valuesFilePath: deployment/environments/staging/chart-values.yaml

[bhack]

@dgageot Seems like kustomize features but for skaffold :wink:

[GeertJohan]

Would like to see this feature as well. Before finding this issue I wasn't sure if it was supported so dived into the code. It seem that this feature could be supported if this case would check for an existing slice item with the same "key":
https://github.com/GoogleContainerTools/skaffold/blob/master/pkg/skaffold/schema/v1alpha3/profiles.go#L115

For slice items in the release slice, they key on which to match and merge items would be the name.

I think the profile should then be written as such:

profiles:
- name: dev
  deploy:
    helm:
      releases:
        - name: my-release < This is matched with an existing release in the non-profile config.
          valuesFilePath: deployment/environments/dev/chart-values.yaml

This does however mean that the profile merger becomes aware of the types of slice field, because the key/value on which basis to merge slice items can be different for difference types of slices.

If no match is found; a new item should be added to the calculated release slice.

[GeertJohan]

Extra idea: Perhaps there should be a way to drop a release from the original config by enabling a profile.
In that case the model suggested by @hypnoglow could be of use, by setting key like this:

- name: staging
  deploy:
    helm:
      releases:
        my-release: {} < This disables/removes the my-release release from the original slice.

However, this requires more changes to the yaml scheme, and I'm not quite sure the use-case is strong enough. I've always thought of profiles as a way to add or change config, not to remove it. Allowing to remove items from the original config allows users to create very complicated skaffold files, so I'm not sure if that should be possible.

[dgageot]

Here's another use case:

@ltouati is trying to use a different Dockerfile for dev and prod. The only way to do that currently is to redefine the whole artifacts sections in a profile.

He ended up using https://github.com/jkozera/j2skaffold

[hypnoglow]

Docker images built by skaffold injected in k8s manifests templated by helm deployed by skaffold templated by jinja. Seems legit 🤪

[balopat]

I'd like to check-in on this issue - do patches in profiles help with these issues or not? @hypnoglow did you try patches?

[hypnoglow]

@balopat Yes, thanks! I'm already using patches and it works fine. I think it is safe to close the issue.

For anyone interested: to solve the original problem, I use patches like this:

profiles:
- name: dev
  patches:
  - op: add
    path: /deploy/helm/releases/0/valuesFiles
    value:
    - deployment/environments/dev/values.yaml