Client-go: [question] can we emulate "kubectl apply" using client-go?

[rdtr]

Hi,

I use client-go and am writing a simple auto deployment code which detects a git push, loads all yaml files which is either newly added or updated then call Update or Create for each yaml file from within a pod in the cluster (Delete is out of my scope at least initially).

At first it looked good but shortly I noticed that Update replaces a whole yaml definition, so some property will be missing after the update such as annoations.kubectl.kubernetes.io/last-applied-configuration. Also I cannot update service without getting resourceVersion and clusterIP from the existing live configuration in advance (these properties are not written in a yaml file).

Our team would still often use kubectl, so I want to make sure that updates from this auto deployment pod doesn't make a conflict with manual kubectl apply command.

I can either GET existing live configuration and merge them with a yaml file I want to apply, or populate a merge request for Patch, but it is not a simple implementation and also reinventing from scratch what kubectl does.
I think then it's easier to install kubectl in a Docker container and run kubectl apply from Go code using exec package, but it's little messy as well.

Is it currently possible to achieve a similar effect as kubectl apply -f xxx.yaml using client-go in a simpler way than I described above?

[ash2k]

See https://github.com/kubernetes/kubernetes/issues/17333

[rdtr]

so it's still in discussion what is the ideal way...

For now I already implemented so that we run kubectl apply -f via Golang from within a pod.
And it works great. So I'm closing this.

[ash2k]

@rdtr You may be interested in our project https://github.com/atlassian/smith

[akutz]

Hi @rdtr,

I think this is being addressed in kubernetes/enhancements#555.

[Jonsy13]

Hi @rdtr Can you point me to where you have implemented the same. I have the same use case for testing.It would be a good help if it is documented somewhere or if you have some resource you can share.

Thanks :)

[akutz]

I wrote this code once upon a time:

import (
    "bufio"
    "bytes"
    "io"

    "github.com/pkg/errors"
    "k8s.io/apimachinery/pkg/apis/meta/v1/unstructured"
    "k8s.io/apimachinery/pkg/runtime"
    utilyaml "k8s.io/apimachinery/pkg/util/yaml"
    "sigs.k8s.io/yaml"
)

// DecodeYAML unmarshals a YAML document or multidoc YAML as unstructured
// objects, placing each decoded object into a channel.
func DecodeYAML(data []byte) (<-chan *unstructured.Unstructured, <-chan error) {

    var (
        chanErr        = make(chan error)
        chanObj        = make(chan *unstructured.Unstructured)
        multidocReader = utilyaml.NewYAMLReader(bufio.NewReader(bytes.NewReader(data)))
    )

    go func() {
        defer close(chanErr)
        defer close(chanObj)

        // Iterate over the data until Read returns io.EOF. Every successful
        // read returns a complete YAML document.
        for {
            buf, err := multidocReader.Read()
            if err != nil {
                if err == io.EOF {
                    return
                }
                chanErr <- errors.Wrap(err, "failed to read yaml data")
                return
            }

            // Do not use this YAML doc if it is unkind.
            var typeMeta runtime.TypeMeta
            if err := yaml.Unmarshal(buf, &typeMeta); err != nil {
                continue
            }
            if typeMeta.Kind == "" {
                continue
            }

            // Define the unstructured object into which the YAML document will be
            // unmarshaled.
            obj := &unstructured.Unstructured{
                Object: map[string]interface{}{},
            }

            // Unmarshal the YAML document into the unstructured object.
            if err := yaml.Unmarshal(buf, &obj.Object); err != nil {
                chanErr <- errors.Wrap(err, "failed to unmarshal yaml data")
                return
            }

            // Place the unstructured object into the channel.
            chanObj <- obj
        }
    }()

    return chanObj, chanErr
}


// ForEachObjectInYAMLActionFunc is a function that is executed against each
// object found in a YAML document.
// When a non-empty namespace is provided then the object is assigned the
// namespace prior to any other actions being performed with or to the object.
type ForEachObjectInYAMLActionFunc func(context.Context, client.Client, *unstructured.Unstructured) error

// ForEachObjectInYAML excutes actionFn for each object in the provided YAML.
// If an error is returned then no further objects are processed.
// The data may be a single YAML document or multidoc YAML.
// When a non-empty namespace is provided then all objects are assigned the
// the namespace prior to any other actions being performed with or to the
// object.
func ForEachObjectInYAML(
    ctx context.Context,
    c client.Client,
    data []byte,
    namespace string,
    actionFn ForEachObjectInYAMLActionFunc) error {

    chanObj, chanErr := DecodeYAML(data)
    for {
        select {
        case obj := <-chanObj:
            if obj == nil {
                return nil
            }
            if namespace != "" {
                obj.SetNamespace(namespace)
            }
            if err := actionFn(ctx, c, obj); err != nil {
                return err
            }
        case err := <-chanErr:
            if err == nil {
                return nil
            }
            return errors.Wrap(err, "received error while decoding yaml")
        }
    }
}

[Jonsy13]

Thanks @akutz , it looks good. Will try to implement it myself.

[penglongli]

Thanks @akutz , it runs fine.

But about this code:

// Define the unstructured object into which the YAML document will be
// unmarshaled.
obj := &unstructured.Unstructured{
    Object: map[string]interface{}{},
}

// Unmarshal the YAML document into the unstructured object.
if err := yaml.Unmarshal(buf, &obj.Object); err != nil {
    chanErr <- errors.Wrap(err, "failed to unmarshal yaml data")
    return
}

I think there have another solution:

import "k8s.io/apimachinery/pkg/runtime/serializer/yaml"

var decUnstructured = yaml.NewDecodingSerializer(unstructured.UnstructuredJSONScheme)

obj := &unstructured.Unstructured{}
// Decode YAML to unstructured object.
if _, _, err := decUnstructured.Decode(buf, nil, obj); err != nil {
    return err
}

Because the obj of the first code just a map. It cannot use obj.GetName() / obj.GetNamespace()
I referenced this article: An_example_of_using_dynamic_client_of_k8s.io/client-go

Dynamic client cannot used to kubernetes cluster which ＜1.16 version. And i make some ugly code to compatible < 1.16 version, there is the demo: penglongli/kubernetes-demo/kubectl-golang