Azure-pipelines-tasks: Interest in "Kubernetes Proxy" functionality?

[hacst]

Required Information

Question, Bug, or Feature?
Type: Question

Enter Task Name: KubernetesV1

Issue Description

One challenge we have encountered when working with Kubernetes from pipelines is that it is hard to run post-deployment integration tests on cluster internal services.

The workaround we have come up with for this is to use the kubectl port-forward feature to make the internal service (or pod) available locally to the build job and then run the tests against that proxy.

I was wondering whether this is functionality for which there was interest upstream? It has some tradeoffs we are willing to take which might not be generally acceptable:

1. To act as a proxy for the rest of the job kubectl port-forward has to keep running after the task completes. This is accomplished by spawning kubectl with the detached option (not possible with normal tool lib but easy with raw child_process). The finalizing task of the job will kill this orphan process without error though, so no user action is needed for cleanup.
2. The port is bound to 127.0.0.1 so while no external access is possible other processes running in the machine can access the port while it exists.
3. To allow kubectl itself select a free local port the task has to parse its log output to figure out which port it actually started to listen on.

Our implementation for this was created based on the KubernetesV1 task. We turned it into a separate "KubernetesProxy" task which only provides the port-forwarding functionality. It takes a target service or pod name and a target service port. The local address of the "proxy" is provided as an output variable.

If there is interest I could create a PR so a closer look can be taken.

[vithati]

@hacst , Presently we didn't see any requests from customers for this enhancement. Azure DevOps can use tasks from market place also. You can implement this as extension and publish in marketplace. Customers can install task from market place if required and use it in their pipeline.

[luzlozanothermo]

Hi Stefan, did you published this task? can you share details of implementation?

[hacst]

@luzlozanothermo I did not publish it as there seemed to be no interest. If this is task would be useful to you I can check whether I can do so. The implementation itself is quite straight-forward.

As mentioned the idea is to spawn kubectl with port-forward and keep it running in the background for future steps in the build. For this we added

    // Spawns kubectl using child_process and forwards the arguments
    public spawnKubectl(args?: string[], options?: child_process.SpawnOptions) : child_process.ChildProcess {
        return child_process.spawn(this.kubectlPath, args, options);
    }

to the ClusterConnection as it knows the kubectl path. That can then be used to start kubectl port-forward in detached mode:

clusterConnection.spawnKubectl(['port-forward', "-n", namespace, target, localPort + ":" + targetPort], {
            detached: true,
            stdio: ['ignore', log, log]
      }
);

Everything around that is just the boilerplate to take in configuration options, figure out which port we actually launched on by tailing the log and so on. As soon as the port is know plugin execution ends and the proxy keeps running.

So basically a fancy kubectl port-forward ... & that can make use of the built-in authentication of the service connections.

The detached process is automatically killed at the end of the build by azure devops. No work needed in the plugin for that.

[danifran]

It is quite useful indeed: we are currently integrating the deployment of a pipeline and the vendor requires to send a couple of consul commands to the consul deployment that is not exposed outside of the cluster. the ability to set-up a port forward proxy on the build agent would allow us to simplify the pipeline a lot