Kubespray: "Error: cluster doesn't provide requestheader-client-ca-file"

[prakashsingh08]

kubernetes version:
$ kubectl version Client Version: version.Info{Major:"1", Minor:"8+", GitVersion:"v1.8.4+coreos.0", GitCommit:"4292f9682595afddbb4f8b1483673449c74f9619", GitTreeState:"clean", BuildDate:"2017-11-21T17:22:25Z", GoVersion:"go1.8.3", Compiler:"gc", Platform:"linux/amd64"} Server Version: version.Info{Major:"1", Minor:"8+", GitVersion:"v1.8.4+coreos.0", GitCommit:"4292f9682595afddbb4f8b1483673449c74f9619", GitTreeState:"clean", BuildDate:"2017-11-21T17:22:25Z", GoVersion:"go1.8.3", Compiler:"gc", Platform:"linux/amd64"}

while running kubectl apply -f demos/monitoring/custom-metrics.yaml as suggested in https://github.com/luxas/kubeadm-workshop#deploying-the-prometheus-operator-for-monitoring-services-in-the-cluster pod is going in CrashLoopBackOff.when o checked pod logs i got error
"Error: cluster doesn't provide requestheader-client-ca-file".

I got solution in below url(i hope this will correct solution)
https://github.com/Azure/acs-engine/pull/1406

but how to implement this. please suggest if possible

[prakashsingh08]

@neith00 , Thanks for your help. suggestion provide by you worked for me.
i added these params in my defaults/main.yml.

• --requestheader-client-ca-file=
• --requestheader-allowed-names=aggregator
• --requestheader-extra-headers-prefix=X-Remote-Extra-
• --requestheader-group-headers=X-Remote-Group
• --requestheader-username-headers=X-Remote-User

please close this issue now.

[NicolasT]

Could you be a bit more precise where this ought to be added? With K8s 1.9, one needs metrics-server to get hpa's to work, and the default way of deploying it (kubectl create -f deploy/ from the metrics-server sources) results in a pod logging this very same thing.

[NicolasT]

Got it to work as well.
For reference, on all master nodes, in /etc/kubernetes/manifests/kube-apiserver.manifest, add the following to the list of command line arguments:

    - --requestheader-client-ca-file=/etc/kubernetes/ssl/ca.pem
    - --requestheader-allowed-names=
    - "--requestheader-extra-headers-prefix=X-Remote-Extra-"
    - --requestheader-group-headers=X-Remote-Group
    - --requestheader-username-headers=X-Remote-User

Then, wait for kubelet to restart the apiserver, and deploy metrics-server as usual.

Note the quotes around one of the lines, took me a while to figure that one out.

Guess this could be added to Kubespray's default config?

[neith00]

You can set them in a var file, for example in all.yml

apiserver_custom_flags:
  - --requestheader-client-ca-file=<path to aggregator CA cert>
  - --requestheader-allowed-names=aggregator

[neith00]

@prakashsingh08 Could you close if it's solved?

[dangpf]

Rancher versions: 1.6.21
docker version: 17.03.3-ce
Kubernetes: 1.10.5
helm: 2.8.2
Environment :
Opened k8s rbac in rancher;
Virtual Machines
Steps: https://svc-cat.io/docs/install/
Results:
apiserver container keeps restarting;
Error:

2018/9/27 下午2:25:30I0927 06:25:30.661826 1 feature_gate.go:194] feature gates: map[OriginatingIdentity:true]
2018/9/27 下午2:25:30I0927 06:25:30.662370 1 feature_gate.go:194] feature gates: map[OriginatingIdentity:true ServicePlanDefaults:false]
2018/9/27 下午2:25:30I0927 06:25:30.662493 1 hyperkube.go:192] Service Catalog version v0.1.32 (built 2018-09-15T01:15:25Z)
2018/9/27 下午2:25:30I0927 06:25:30.662528 1 run_server.go:59] Preparing to run API server
2018/9/27 下午2:25:36I0927 06:25:36.841275 1 round_trippers.go:386] curl -k -v -XGET -H "Authorization: Bearer eyJhbGciOiJSUzI1NiIsImtpZCI6IiJ9.eyJpc3MiOiJrdWJlcm5ldGVzL3NlcnZpY2VhY2NvdW50Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9uYW1lc3BhY2UiOiJjYXRhbG9nIiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9zZWNyZXQubmFtZSI6InNlcnZpY2UtY2F0YWxvZy1hcGlzZXJ2ZXItdG9rZW4tdGJqbnoiLCJrdWJlcm5ldGVzLmlvL3NlcnZpY2VhY2NvdW50L3NlcnZpY2UtYWNjb3VudC5uYW1lIjoic2VydmljZS1jYXRhbG9nLWFwaXNlcnZlciIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VydmljZS1hY2NvdW50LnVpZCI6ImE0MGI2ZTQ4LWMxMjctMTFlOC05ZDEzLTAyMTU3MjNmMmM4ZSIsInN1YiI6InN5c3RlbTpzZXJ2aWNlYWNjb3VudDpjYXRhbG9nOnNlcnZpY2UtY2F0YWxvZy1hcGlzZXJ2ZXIifQ.kXzApv8nhOYHiRS9cGwhC4ryUDh2_BlDAH_GwaJ5dbg3uEFM9ntI29lLEbeP1nvDN6Ja42BPndWPZEHzRGVX4ZZlU4YdwpLL8EssiCZ7jyxhE3e8Kfn8PBNrkVszovyEz5M4jSBpWI_4brFh0-j8TqZya4LRLoeW7-ZUTb71qu1vXH05F9MevFjzmrNZ8kTtKAbrXdMrMqPjgjjnm5kq8ZXb-i3qYgnmCULzwATJmNymgY2lXr0FV_dV078OZUtaJqynbOhqyMN_uwW6hAvxXppw3AWSOpgvoOvIAWL6wLiUu_adCAiNivcW4ar9fQNBHk5fiYRZusL2RMsMx8WC6g" -H "Accept: application/json, /" -H "User-Agent: service-catalog/v0.1.32 (linux/amd64) kubernetes/3d1d270" 'https://10.43.0.1:443/api/v1/namespaces/kube-system/configmaps/extension-apiserver-authentication'
2018/9/27 下午2:25:36I0927 06:25:36.862606 1 round_trippers.go:405] GET https://10.43.0.1:443/api/v1/namespaces/kube-system/configmaps/extension-apiserver-authentication 200 OK in 21 milliseconds
2018/9/27 下午2:25:36I0927 06:25:36.862623 1 round_trippers.go:411] Response Headers:
2018/9/27 下午2:25:36I0927 06:25:36.862627 1 round_trippers.go:414] Date: Thu, 27 Sep 2018 06:25:36 GMT
2018/9/27 下午2:25:36I0927 06:25:36.862630 1 round_trippers.go:414] Content-Type: application/json
2018/9/27 下午2:25:36I0927 06:25:36.862633 1 round_trippers.go:414] Content-Length: 1390
2018/9/27 下午2:25:36I0927 06:25:36.862888 1 request.go:897] Response Body: {"kind":"ConfigMap","apiVersion":"v1","metadata":{"name":"extension-apiserver-authentication","namespace":"kube-system","selfLink":"/api/v1/namespaces/kube-system/configmaps/extension-apiserver-authentication","uid":"2a2472b3-b63c-11e8-8e37-021572b1e491","resourceVersion":"49","creationTimestamp":"2018-09-12T03:30:15Z"},"data":{"client-ca-file":"-----BEGIN CERTIFICATE-----\nMIICxjCCAa6gAwIBAgIHXHC7KmSvfTANBgkqhkiG9w0BAQ0FADARMQ8wDQYDVQQK\nDAZjYXR0bGUwHhcNMTgwOTEyMDMyMzA0WhcNMjgwOTA5MDMyMzA0WjARMQ8wDQYD\nVQQKDAZjYXR0bGUwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQDXvrjF\nJyNjNIRZQTvvLW0l7JkrLv7d3oGEWumX+D/oLyDaQ3PMmV2sbzz7avXclx2JJVTO\nmHSx3iEgNfoUU1eJK8VWt/QOaqfPvEBY5LC2MDjpNeAkrZq7FUgkKavC9las8q9c\n+U7aYrjoqi1tzKdS0TCDVHlFE+VLvCafuY2UTYxabhcQuR6KJ5SLz6CuFXqveqxh\n82DsNk3CjiHvMzlVpa1zCD9Xe59As1Cdej6NGineKHQNrS9X8EhwD2s+w1Bs+J8n\neyyXKEucyrT6GD56Gih0051m/kDnYcaGqN2NKEI6Q2bFMO2Eoqisx8FaVU6UvTbS\njlbYeYjqy0sTg9O7AgMBAAGjIzAhMA8GA1UdEwEB/wQFMAMBAf8wDgYDVR0PAQH/\nBAQDAgKkMA0GCSqGSIb3DQEBDQUAA4IBAQDJsxo8sZbh1jrSykK3mXAP5HJJhsAK\nDSLRU4w90S1bBkKSPth7o/DisUuFx9qJKWovQ++2XT83KE73/YLJnw3I2osrvUeM\nPMi6OexbUZpX7vSi3jK2WE6dwv3lsLVFw4QKTxZu8SZ73eYv3ttx6ug9+mpS69hF\nZUbXD5Z0png1uTXI7FIFF5+/vK2jrPPTFPRIhV1vBP+qWvsQs1Bk7mEpSk+aseOV\niuq9gJwLadL8wvvHRQfusfYPQ5BlDqQAL5e7zaV3/mb+JtR9e2lrgIEwpJyXKHsy\nWMAftbFERT3/n9nei8tPz5bryfJjeR1ogLTD3N17w20WfN5fEloWD1Ly\n-----END CERTIFICATE-----\n"}}
2018/9/27 下午2:25:36I0927 06:25:36.863915 1 round_trippers.go:386] curl -k -v -XGET -H "Accept: application/json, /" -H "User-Agent: service-catalog/v0.1.32 (linux/amd64) kubernetes/3d1d270" -H "Authorization: Bearer eyJhbGciOiJSUzI1NiIsImtpZCI6IiJ9.eyJpc3MiOiJrdWJlcm5ldGVzL3NlcnZpY2VhY2NvdW50Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9uYW1lc3BhY2UiOiJjYXRhbG9nIiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9zZWNyZXQubmFtZSI6InNlcnZpY2UtY2F0YWxvZy1hcGlzZXJ2ZXItdG9rZW4tdGJqbnoiLCJrdWJlcm5ldGVzLmlvL3NlcnZpY2VhY2NvdW50L3NlcnZpY2UtYWNjb3VudC5uYW1lIjoic2VydmljZS1jYXRhbG9nLWFwaXNlcnZlciIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VydmljZS1hY2NvdW50LnVpZCI6ImE0MGI2ZTQ4LWMxMjctMTFlOC05ZDEzLTAyMTU3MjNmMmM4ZSIsInN1YiI6InN5c3RlbTpzZXJ2aWNlYWNjb3VudDpjYXRhbG9nOnNlcnZpY2UtY2F0YWxvZy1hcGlzZXJ2ZXIifQ.kXzApv8nhOYHiRS9cGwhC4ryUDh2_BlDAH_GwaJ5dbg3uEFM9ntI29lLEbeP1nvDN6Ja42BPndWPZEHzRGVX4ZZlU4YdwpLL8EssiCZ7jyxhE3e8Kfn8PBNrkVszovyEz5M4jSBpWI_4brFh0-j8TqZya4LRLoeW7-ZUTb71qu1vXH05F9MevFjzmrNZ8kTtKAbrXdMrMqPjgjjnm5kq8ZXb-i3qYgnmCULzwATJmNymgY2lXr0FV_dV078OZUtaJqynbOhqyMN_uwW6hAvxXppw3AWSOpgvoOvIAWL6wLiUu_adCAiNivcW4ar9fQNBHk5fiYRZusL2RMsMx8WC6g" 'https://10.43.0.1:443/api/v1/namespaces/kube-system/configmaps/extension-apiserver-authentication'
2018/9/27 下午2:25:36I0927 06:25:36.872171 1 round_trippers.go:405] GET https://10.43.0.1:443/api/v1/namespaces/kube-system/configmaps/extension-apiserver-authentication 200 OK in 8 milliseconds
2018/9/27 下午2:25:36I0927 06:25:36.872188 1 round_trippers.go:411] Response Headers:
2018/9/27 下午2:25:36I0927 06:25:36.872192 1 round_trippers.go:414] Content-Type: application/json
2018/9/27 下午2:25:36I0927 06:25:36.872195 1 round_trippers.go:414] Content-Length: 1390
2018/9/27 下午2:25:36I0927 06:25:36.872197 1 round_trippers.go:414] Date: Thu, 27 Sep 2018 06:25:36 GMT
2018/9/27 下午2:25:36I0927 06:25:36.872503 1 request.go:897] Response Body: {"kind":"ConfigMap","apiVersion":"v1","metadata":{"name":"extension-apiserver-authentication","namespace":"kube-system","selfLink":"/api/v1/namespaces/kube-system/configmaps/extension-apiserver-authentication","uid":"2a2472b3-b63c-11e8-8e37-021572b1e491","resourceVersion":"49","creationTimestamp":"2018-09-12T03:30:15Z"},"data":{"client-ca-file":"-----BEGIN CERTIFICATE-----\nMIICxjCCAa6gAwIBAgIHXHC7KmSvfTANBgkqhkiG9w0BAQ0FADARMQ8wDQYDVQQK\nDAZjYXR0bGUwHhcNMTgwOTEyMDMyMzA0WhcNMjgwOTA5MDMyMzA0WjARMQ8wDQYD\nVQQKDAZjYXR0bGUwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQDXvrjF\nJyNjNIRZQTvvLW0l7JkrLv7d3oGEWumX+D/oLyDaQ3PMmV2sbzz7avXclx2JJVTO\nmHSx3iEgNfoUU1eJK8VWt/QOaqfPvEBY5LC2MDjpNeAkrZq7FUgkKavC9las8q9c\n+U7aYrjoqi1tzKdS0TCDVHlFE+VLvCafuY2UTYxabhcQuR6KJ5SLz6CuFXqveqxh\n82DsNk3CjiHvMzlVpa1zCD9Xe59As1Cdej6NGineKHQNrS9X8EhwD2s+w1Bs+J8n\neyyXKEucyrT6GD56Gih0051m/kDnYcaGqN2NKEI6Q2bFMO2Eoqisx8FaVU6UvTbS\njlbYeYjqy0sTg9O7AgMBAAGjIzAhMA8GA1UdEwEB/wQFMAMBAf8wDgYDVR0PAQH/\nBAQDAgKkMA0GCSqGSIb3DQEBDQUAA4IBAQDJsxo8sZbh1jrSykK3mXAP5HJJhsAK\nDSLRU4w90S1bBkKSPth7o/DisUuFx9qJKWovQ++2XT83KE73/YLJnw3I2osrvUeM\nPMi6OexbUZpX7vSi3jK2WE6dwv3lsLVFw4QKTxZu8SZ73eYv3ttx6ug9+mpS69hF\nZUbXD5Z0png1uTXI7FIFF5+/vK2jrPPTFPRIhV1vBP+qWvsQs1Bk7mEpSk+aseOV\niuq9gJwLadL8wvvHRQfusfYPQ5BlDqQAL5e7zaV3/mb+JtR9e2lrgIEwpJyXKHsy\nWMAftbFERT3/n9nei8tPz5bryfJjeR1ogLTD3N17w20WfN5fEloWD1Ly\n-----END CERTIFICATE-----\n"}}
2018/9/27 下午2:25:36Error: cluster doesn't provide requestheader-client-ca-file

[fejta-bot]

Stale issues rot after 30d of inactivity.
Mark the issue as fresh with /remove-lifecycle rotten.
Rotten issues close after an additional 30d of inactivity.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle rotten

[fejta-bot]

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen.
Mark the issue as fresh with /remove-lifecycle rotten.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/close

[k8s-ci-robot]

@fejta-bot: Closing this issue.

In response to this:

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen.
Mark the issue as fresh with /remove-lifecycle rotten.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.