Skaffold: Question: Is there a way to template the kaniko pod?

[adjavaherian]

As far as I can tell from the docs, there's not a way to specify a template that Skaffold can use when creating the Kaniko pod. Am I missing something? For example, in a Tekton pipeline, I use Skaffold as a container (task step) which then itself spawns a Kaniko pod. I want to mount a volume into the Kaniko pod so I can capture some bytes for use in the original task pod. It looks like you can mount a volume no problem, but the Kaniko volume spec in the skaffold.yaml is also not well documented outside of the secret mount. Any help or an example would be great, Thanks.

[tstromberg]

@tejal29 - is this something you can assistance on?

[tejal29]

@adjavaherian currently it is not possible to template kaniko pod spec within skaffold.yaml.
Skaffold interfaces with many tools and we have thought in past about

1. how can we add generics fields to pass as arguments to these tools instead of add a config field for each argument.
   We have done that by adding key value map like buildArgs, args in the spec.
   Kaniko is special since it has spec and we don't have to add a way for users to specify a spec. something we can look into if its asked for a lot.

Re: Volumemounts.
The volumeMounts field in Kaniko volume spec is same is k8 volume spec.
e.g.

volumeMounts: []v1.VolumeMount{
            {
                Name:      "cm-volume-1",
                ReadOnly:  true,
                MountPath: "/cm-test-mount-path",
                SubPath:   "/subpath",
            },
            {
                Name:      "secret-volume-1",
                ReadOnly:  true,
                MountPath: "/secret-test-mount-path",
                SubPath:   "/subpath",
            },
        },

[frayer]

I'm interested in something like this as well, but get schema errors in Skaffold v1.8.0. I would like to use gcr.io/kaniko-project/warmer:latest as shown in this example to create a volume with the cached base image I'll be building on top of. That could be a step outside of Skaffold, but once created I would like to mount that volume and use the --cache-dir Kaniko flag to use it at runtime.

Going off the volumes and volumeMounts fields mentioned in the docs, I tried the following:

apiVersion: skaffold/v2beta2
kind: Config
metadata:
  name: my-app
build:
  artifacts:
  - image: repo/my-app
    kaniko:
      cache:
        repo: repo/my-app-cache
      volumeMounts:
      - name: kaniko-cache
        mountPath: /cache
  cluster:
    dockerConfig:
      path: ./config.json
    volumes:
    - name: kaniko-cache
      persistentVolumeClaim:
        claimName: kaniko-cache-claim

I get a parsing error on both though:

> skaffold build
FATA[0000] creating runner: parsing skaffold config: unable to parse config: yaml: unmarshal errors:
  line 9: field mountPath not found in type v1.VolumeMount
  line 16: field persistentVolumeClaim not found in type v1.Volume

[dranes]

Getting the exact same error

parsing skaffold config: unable to parse config: yaml: unmarshal errors:
  line 11: field mountPath not found in type v1.VolumeMount

[tejal29]

@frayer and @adjavaherian As @dranes mentioned in #4175 the right way to configure volumeMounts is to use lower case mountpath and readonly

    kaniko:
      volumeMounts: 
      - name: data
        mountpath: /data    
        readonly: true  

[tejal29]

Closing this issue now. Please comment if it does not fix your issue