Linkerd2: Add check for Tap APIService

[Pothulapati]

Feature Request

What problem are you trying to solve?

Linkerd check is very useful rn as it does comprehensive tests on most of the control-plane integration points. It would be really great to have one for the Tap APIService too, as we have been seeing some issues around APIServices on some clusters.

How should the problem be solved?

We may check for

• Use k8s API and check the status of the APIService
• Talk to the APIService and see if it fails

What do you want to happen? Add any considered drawbacks.

.. to make Linkerd check more awesome?

Any alternatives you've considered?

Is there another way to solve this problem that isn't as good a solution?

How would users interact with this feature?

Users can use the check command and also see the status of the APIService

[hasheddan]

Happy to take this one!

[grampelberg]

That'd be amazing!

[droidnoob]

Is @hasheddan working on it. If not I can take it up. @grampelberg

[hasheddan]

@droidnoob I fell behind on this, feel free to take this on if you have bandwidth! Thanks :)

[droidnoob]

Great.
I'm new at this. Where do I start?
@Pothulapati @grampelberg

[Pothulapati]

Hey @droidnoob
Thanks for taking the effort!

The plan is to add more checks for the healthchecker that we already have here https://github.com/linkerd/linkerd2/blob/92532bc0b31b082be42ee035a3c4a974fa2c8693/pkg/healthcheck/healthcheck.go#L536

I took a look for the code to access ApiService, This turned to be a bit harder than expected, as I don't see a way to use client-go to get apiregistrations.APIService, that client is instead part of the kube-aggregator project. So, the clientset code has to be written like this

diff --git a/pkg/healthcheck/healthcheck.go b/pkg/healthcheck/healthcheck.go
index 21f38c27..c4c7cb9a 100644
--- a/pkg/healthcheck/healthcheck.go
+++ b/pkg/healthcheck/healthcheck.go
@@ -29,6 +29,7 @@ import (
        "k8s.io/apimachinery/pkg/runtime/schema"
        k8sVersion "k8s.io/apimachinery/pkg/version"
        "k8s.io/client-go/kubernetes"
+       apiregistrationv1client "k8s.io/kube-aggregator/pkg/client/clientset_generated/clientset/typed/apiregistration/v1"
 )

 // CategoryID is an identifier for the types of health checks.
@@ -1348,6 +1349,21 @@ func (hc *HealthChecker) checkCanCreate(namespace, group, version, resource stri
        )
 }

+func (hc *HealthChecker) checkAPIService(name string) error {
+
+       apiServiceClient, err := apiregistrationv1client.NewForConfig(hc.kubeAPI.Config)
+       if err != nil {
+               return fmt.Errorf("unexpected error: Couldn't initialise client")
+       }
+
+       apiService, err := apiServiceClient.APIServices().Get(name, metav1.GetOptions{})
+       if err != nil {
+               return fmt.Errorf("unexpected error: Couldn't get the service")
+       }
+
+       apiService.Status
+}
+
 func (hc *HealthChecker) checkCapability(cap string) error {
        if hc.kubeAPI == nil {
                // we should never get here

We can use that apiService object and then check the status.
This check would be included as part of the health-checker, that is linkerd first.

One more check I wthink would be to, see if the endpoint is actually replying a 200 or not.
Maybe @ihcsim can add more here.

Feel free to get back if you have any issues :)

[droidnoob]

Thanks for the info. @Pothulapati
It was really helpful

[droidnoob]

@Pothulapati I have built it using the instructions in BUILD.md. The linkerd-identity pod seems to be crashed

NAMESPACE     NAME                                      READY   STATUS             RESTARTS   AGE
kube-system   coredns-5644d7b6d9-42pw7                  1/1     Running            0          64m
kube-system   coredns-5644d7b6d9-c2t5v                  1/1     Running            0          64m
kube-system   etcd-minikube                             1/1     Running            0          64m
kube-system   kube-addon-manager-minikube               1/1     Running            0          64m
kube-system   kube-apiserver-minikube                   1/1     Running            0          63m
kube-system   kube-controller-manager-minikube          1/1     Running            0          63m
kube-system   kube-proxy-g86tq                          1/1     Running            0          64m
kube-system   kube-scheduler-minikube                   1/1     Running            0          63m
kube-system   storage-provisioner                       1/1     Running            0          64m
linkerd       linkerd-controller-698bcbb785-dkckh       2/3     Running            0          4m5s
linkerd       linkerd-destination-7786547f79-rj57c      1/2     Running            0          4m5s
linkerd       linkerd-grafana-69d48589fc-lwcnf          1/2     Running            0          4m4s
linkerd       linkerd-identity-684cfcb5b6-tw5xr         0/2     CrashLoopBackOff   5          4m6s
linkerd       linkerd-prometheus-7d99cc55f8-2tkkl       1/2     Running            0          4m4s
linkerd       linkerd-proxy-injector-744f8f649f-gg227   1/2     Running            0          4m4s
linkerd       linkerd-sp-validator-57ccf4944d-wlbwt     1/2     Running            0          4m4s
linkerd       linkerd-tap-7c84789f4f-895vk              1/2     Running            0          4m3s
linkerd       linkerd-web-5b64dcb696-v6dpw              1/2     Running            0          4m4s

[Pothulapati]

Hey,
Can you jump onto the Linkerd slack, so that we can keep the issue focused to that? :)