skaffold.yaml schema generation

[etanshaul]

This is to start the discussion around generating a schema for skaffold.yaml consumable by other tools like IDEs and editors.

In google-container-tools-intellij we are building out editing support for Skaffold. This includes things autocomplete, file inspections etc. To do this in a robust way (keeping up with schema changes and so on), we'd like to have a versioned schema that ideally is auto-generated, possibly using a Go struct to JSON schema generator.

As an example, the IntelliJ Kubernetes plugin currently provides support for editing Kubernetes manifests. It accomplishes this by consuming the published k8s openapi-spec. Within this spec, are definitions (in the form of JSON schema) such as the following Deployment schema:

   "io.k8s.api.apps.v1.Deployment": {
    "description": "Deployment enables declarative updates for Pods and ReplicaSets.",
    "properties": {
     "apiVersion": {
      "description": "APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#resources",
      "type": "string"
     },
     "kind": {
      "description": "Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#types-kinds",
      "type": "string"
     },
     "metadata": {
      "description": "Standard object metadata.",
      "$ref": "#/definitions/io.k8s.apimachinery.pkg.apis.meta.v1.ObjectMeta"
     },
     "spec": {
      "description": "Specification of the desired behavior of the Deployment.",
      "$ref": "#/definitions/io.k8s.api.apps.v1.DeploymentSpec"
     },
     "status": {
      "description": "Most recently observed status of the Deployment.",
      "$ref": "#/definitions/io.k8s.api.apps.v1.DeploymentStatus"
     }
    },
    "x-kubernetes-group-version-kind": [
     {
      "group": "apps",
      "kind": "Deployment",
      "version": "v1"
     }
    ]
   }

This spec is consumed by the plugin to provide rich editing support.

Producing a JSON schema like this for skaffold.yaml would allow us to create the same level of editor support. (Note there is some extra metadata in there not currently in the Skaffold Go structs, like description which would be nice to have as well).

Any thoughts, concerns?

[balopat]

I believe it's important to enable rich editing support for discoverability purposes.
Also publishing a schema in theory should be possible from our go structs describing the pipeline.

Concern: It's a little bit weird to use JSON Schema or Open API given that they are meant for describing RESTful APIs, but I don't have a better suggestion. Maybe, if I squint, we could pass in a "skaffold yaml" message to a managed skaffold service as part of a "setup" command...?

Living with this semantic dissonance for me is well justified for the value that we'd get out of IDE integration. @dgageot @r2d4 any concerns?

[etanshaul]

@balopat I totally would share your concern about Open API. But given that we just need a JSON schema (which has no ties to RESTful API definition), does that alleviate the concern? Maybe I missed your point?

To clarify for those who haven't reviewed the internal doc: The example I mention about Open API is just an implementation detail used by the IntelliJ Kubernetes plugin. The actual validation comes from a JSON Schema (that happens to be contained within the OpenApi spec).

[redborian]

Not sure if this is helpful, but I wrote a tool to convert the go yaml-annotated types to json schema.
More details here: https://github.com/redborian/go-yaml-to-jsonschema

[balopat]

Is there any work left here or can we close this?

[dgageot]

Let's close it

[etanshaul]

yep. thanks for the help with this @balopat and @dgageot

[redborian]

For reference, the generated schema is located here: https://github.com/GoogleContainerTools/skaffold/tree/master/docs/content/en/schemas