Linkerd2: Go 1.15: Running linkerd on 1.19 results in Certificate error

[balchua]

Bug Report

What is the issue?

Running linkerd tap on kubernetes 1.19 throws this error.

unexpected API response: error trying to reach service: x509: certificate relies on legacy Common Name field, use SANs or temporarily enable Common Name matching with GODEBUG=x509ignoreCN=0

Using the dashboard > Live Calls also shows the error.

How can it be reproduced?

Install linkerd on Kubernetes 1.19

Logs, error output, etc

unexpected API response: error trying to reach service: x509: certificate relies on legacy Common Name field, use SANs or temporarily enable Common Name matching with GODEBUG=x509ignoreCN=0

linkerd check output

kubernetes-api
--------------
√ can initialize the client
√ can query the Kubernetes API

kubernetes-version
------------------
√ is running the minimum Kubernetes API version
√ is running the minimum kubectl version

linkerd-existence
-----------------
√ 'linkerd-config' config map exists
√ heartbeat ServiceAccount exist
√ control plane replica sets are ready
√ no unschedulable pods
√ controller pod is running
√ can initialize the client
√ can query the control plane API

linkerd-config
--------------
√ control plane Namespace exists
√ control plane ClusterRoles exist
√ control plane ClusterRoleBindings exist
√ control plane ServiceAccounts exist
√ control plane CustomResourceDefinitions exist
√ control plane MutatingWebhookConfigurations exist
√ control plane ValidatingWebhookConfigurations exist
√ control plane PodSecurityPolicies exist

linkerd-identity
----------------
√ certificate config is valid
√ trust anchors are using supported crypto algorithm
√ trust anchors are within their validity period
√ trust anchors are valid for at least 60 days
√ issuer cert is using supported crypto algorithm
√ issuer cert is within its validity period
√ issuer cert is valid for at least 60 days
√ issuer cert is issued by the trust anchor

linkerd-identity-data-plane
---------------------------
√ data plane proxies certificate match CA

linkerd-api
-----------
√ control plane pods are ready
√ control plane self-check
√ [kubernetes] control plane can talk to Kubernetes
√ [prometheus] control plane can talk to Prometheus
√ tap api service is running

linkerd-version
---------------
√ can determine the latest version
‼ cli is up-to-date
    is running version 2.8.0 but the latest stable version is 2.8.1
    see https://linkerd.io/checks/#l5d-version-cli for hints

linkerd-data-plane
------------------
√ data plane namespace exists
√ data plane proxies are ready
√ data plane proxy metrics are present in Prometheus
‼ data plane is up-to-date
    Some data plane pods are not running the current version:
    * linkerd/linkerd-destination-767bcfbf4-xxqxq (stable-2.8.0)
    * linkerd/linkerd-prometheus-6699dfbc44-tcq9j (stable-2.8.0)
    * linkerd/linkerd-sp-validator-d5fc9845c-ggknv (stable-2.8.0)
    * linkerd/linkerd-tap-6b9dc77844-4kpz5 (stable-2.8.0)
    * linkerd/linkerd-identity-8585d57c49-fqzmm (stable-2.8.0)
    * linkerd/linkerd-controller-5d6bbdb8b5-4j4d9 (stable-2.8.0)
    * linkerd/linkerd-web-67db674d86-lmd4l (stable-2.8.0)
    * linkerd/linkerd-proxy-injector-9bd885986-d67wj (stable-2.8.0)
    * linkerd/linkerd-grafana-84d56f5794-btq4t (stable-2.8.0)
    see https://linkerd.io/checks/#l5d-data-plane-version for hints
√ data plane and cli versions match

linkerd-addons
--------------
√ 'linkerd-config-addons' config map exists

linkerd-grafana
---------------
√ grafana add-on service account exists
√ grafana add-on config map exists
√ grafana pod is running

Status check results are √

Environment

• Kubernetes Version: 1.19
• Cluster Environment: MicroK8s
• Host OS: Ubuntu 20.04
• Linkerd version: 2.8.0

[Pothulapati]

Working on this currently. Looks like it was because K8s updated its go version to 1.15 (i.e latest) which changed how it reads certificates https://github.com/golang/go/issues/39568#issuecomment-671424481

[olix0r]

It looks http://masterminds.github.io/sprig/crypto.html references a new function, genSelfSignedCert can replace genCA to support alternateDNS names

[Pothulapati]

4919 address this issue, and it makes tap, etc work on kubernetes 1.19

But if we updated our linkerd2 repo, and images to use go 1.15, the identity component fails to startup as the trustanchorCerts we generate in the CLI, don't set alternateDNS names and hence the identity component fails to startup with the same error above.
Updating the binary to generate certs with alternateDNS fields should be the solution there, and should be a separate PR I guess.

[balchua]

I've tested the latest edge 20.9.1 all good. I am just wondering if this is also coming to 2.8?
Thanks

[Pothulapati]

@balchua It should land in 2.9, which is planned for early October!

[balchua]

Ok thanks.