Autoscaler: vpa-admission-controller- not created

[Deepak-Vohra]

1. Installed metrics server.

C:\EKS>kubectl -n kube-system get deployment/metrics-server
NAME             READY   UP-TO-DATE   AVAILABLE   AGE
metrics-server   1/1     1            1           105m

1. Downloaded autoscaler.

C:\EKS>git clone https://github.com/kubernetes/autoscaler.git
Cloning into 'autoscaler'...
remote: Enumerating objects: 1, done.
remote: Counting objects: 100% (1/1), done.
remote: Total 95628 (delta 0), reused 0 (delta 0), pack-reused 95627R
Receiving objects: 100% (95628/95628), 94.50 MiB | 2.05 MiB/s, done.

Resolving deltas: 100% (60577/60577), done.
Updating files: 100% (20298/20298), done.

1. Installed VPA.

C:\EKS>kubectl apply -f  autoscaler\vertical-pod-autoscaler\deploy\
deployment.apps/vpa-admission-controller created
service/vpa-webhook created
serviceaccount/vpa-recommender created
deployment.apps/vpa-recommender created
serviceaccount/vpa-updater created
deployment.apps/vpa-updater created
customresourcedefinition.apiextensions.k8s.io/verticalpodautoscalers.autoscaling
.k8s.io created
customresourcedefinition.apiextensions.k8s.io/verticalpodautoscalercheckpoints.a
utoscaling.k8s.io created
customresourcedefinition.apiextensions.k8s.io/verticalpodautoscalers.autoscaling
.k8s.io configured
customresourcedefinition.apiextensions.k8s.io/verticalpodautoscalercheckpoints.a
utoscaling.k8s.io configured
customresourcedefinition.apiextensions.k8s.io/verticalpodautoscalers.poc.autosca
ling.k8s.io created
customresourcedefinition.apiextensions.k8s.io/verticalpodautoscalercheckpoints.p
oc.autoscaling.k8s.io created
clusterrole.rbac.authorization.k8s.io/system:metrics-reader created
clusterrole.rbac.authorization.k8s.io/system:vpa-actor created
clusterrole.rbac.authorization.k8s.io/system:vpa-checkpoint-actor created
clusterrole.rbac.authorization.k8s.io/system:evictioner created
clusterrolebinding.rbac.authorization.k8s.io/system:metrics-reader created
clusterrolebinding.rbac.authorization.k8s.io/system:vpa-actor created
clusterrolebinding.rbac.authorization.k8s.io/system:vpa-checkpoint-actor created

clusterrole.rbac.authorization.k8s.io/system:vpa-target-reader created
clusterrolebinding.rbac.authorization.k8s.io/system:vpa-target-reader-binding cr
eated
clusterrolebinding.rbac.authorization.k8s.io/system:vpa-evictionter-binding crea
ted
serviceaccount/vpa-admission-controller created
clusterrole.rbac.authorization.k8s.io/system:vpa-admission-controller created
clusterrolebinding.rbac.authorization.k8s.io/system:vpa-admission-controller cre
ated
clusterrole.rbac.authorization.k8s.io/system:vpa-status-reader created
clusterrolebinding.rbac.authorization.k8s.io/system:vpa-status-reader-binding cr
eated
customresourcedefinition.apiextensions.k8s.io/verticalpodautoscalers.autoscaling
.k8s.io configured
customresourcedefinition.apiextensions.k8s.io/verticalpodautoscalercheckpoints.a
utoscaling.k8s.io configured

1. VPA Admission Controller does not get created.

C:\EKS>kubectl get pods -n kube-system
NAME                                        READY   STATUS              RESTARTS
   AGE
aws-node-6nkfq                              1/1     Running             0
   179m
aws-node-6wtjz                              1/1     Running             0
   177m
aws-node-wnqnc                              1/1     Running             0
   175m
coredns-56678dcf76-77xhp                    1/1     Running             0
   176m
coredns-56678dcf76-f44cw                    1/1     Running             0
   3h
kube-proxy-87f6m                            1/1     Running             0
   179m
kube-proxy-qt8f7                            1/1     Running             0
   175m
kube-proxy-z5w84                            1/1     Running             0
   177m
metrics-server-7fcf9cc98b-lczwp             1/1     Running             0
   134m
vpa-admission-controller-69bf4dd698-hhd9b   0/1     ContainerCreating   0
   39s
vpa-recommender-6cf9fd5c45-9hwlx            1/1     Running             0
   44s
vpa-updater-7d9db6565c-xgfzj                1/1     Running             0
   43s

[bskiba]

Can you paste the result of
kubectl describe pod -n kube-system vpa-admission-controller-69bf4dd698-hhd9b

[Deepak-Vohra]

Is any other configuration needed than discussed at https://docs.aws.amazon.com/eks/latest/userguide/vertical-pod-autoscaler.html?

Requested output is as follows.

C:\EKS>kubectl describe pod -n kube-system vpa-admission-controller-69bf4dd698-q
dj76
Name:           vpa-admission-controller-69bf4dd698-qdj76
Namespace:      kube-system
Priority:       0
Node:           ip-192-168-88-36.ec2.internal/192.168.88.36
Start Time:     Mon, 10 Feb 2020 14:37:31 -0800
Labels:         app=vpa-admission-controller
                pod-template-hash=69bf4dd698
Annotations:    kubernetes.io/psp: eks.privileged
Status:         Pending
IP:
IPs:            <none>
Controlled By:  ReplicaSet/vpa-admission-controller-69bf4dd698
Containers:
  admission-controller:
    Container ID:
    Image:          k8s.gcr.io/vpa-admission-controller:0.6.3
    Image ID:
    Port:           8000/TCP
    Host Port:      0/TCP
    State:          Waiting
      Reason:       ContainerCreating
    Ready:          False
    Restart Count:  0
    Limits:
      cpu:     200m
      memory:  500Mi
    Requests:
      cpu:     50m
      memory:  200Mi
    Environment:
      NAMESPACE:  kube-system (v1:metadata.namespace)
    Mounts:
      /etc/tls-certs from tls-certs (ro)
      /var/run/secrets/kubernetes.io/serviceaccount from vpa-admission-controlle
r-token-ng8tz (ro)
Conditions:
  Type              Status
  Initialized       True
  Ready             False
  ContainersReady   False
  PodScheduled      True
Volumes:
  tls-certs:
    Type:        Secret (a volume populated by a Secret)
    SecretName:  vpa-tls-certs
    Optional:    false
  vpa-admission-controller-token-ng8tz:
    Type:        Secret (a volume populated by a Secret)
    SecretName:  vpa-admission-controller-token-ng8tz
    Optional:    false
QoS Class:       Burstable
Node-Selectors:  <none>
Tolerations:     node.kubernetes.io/not-ready:NoExecute for 300s
                 node.kubernetes.io/unreachable:NoExecute for 300s
Events:
  Type     Reason       Age                  From
     Message
  ----     ------       ----                 ----
     -------
  Normal   Scheduled    2m18s                default-scheduler
     Successfully assigned kube-system/vpa-admission-controller-69bf4dd698-qdj76
 to ip-192-168-88-36.ec2.internal
  Warning  FailedMount  74s (x8 over 2m18s)  kubelet, ip-192-168-88-36.ec2.inter
nal  MountVolume.SetUp failed for volume "tls-certs" : secrets "vpa-tls-certs" n
ot found

C:EKS>

[bskiba]

Is any other configuration needed than discussed at https://docs.aws.amazon.com/eks/latest/userguide/vertical-pod-autoscaler.html?

This is a question to the author of that documentation, it was not created by SIG Autoscaling

To the original problem. From the logs:

kubelet, ip-192-168-88-36.ec2.internal MountVolume.SetUp failed for volume "tls-certs" : secrets "vpa-tls-certs" not found

The vpa-tls-certs secret is created by gencerts.sh script. You created the VPA by
kubectl apply -f autoscaler\vertical-pod-autoscaler\deploy\

This does not generate the needed certs. Both the docs you mention and documentation in this repo make the installation step
./hack/vpa-up.sh
You need to either use this script to deploy VPA or generate the certs yourself (for example by running gencerts.sh)

[bskiba]

/close
I'm closing the issue, please reopen if the instruction from previous comment doesn't help

[k8s-ci-robot]

@bskiba: Closing this issue.

In response to this:

/close
I'm closing the issue, please reopen if the instruction from previous comment doesn't help

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[Deepak-Vohra]

While AWS EKS is very much supported on Windows, the .sh scripts that generate the certs for the VPA don't seem to be favorable to Windows. Installed Git Bash to run the .sh scripts, but still getting :

dvohra@dvohra-PC MINGW64 /c/EKS/autoscaler/vertical-pod-autoscaler/hack
$ sh vpa-up.sh
Generating certs for the VPA Admission Controller in /tmp/vpa-certs.
Generating RSA private key, 2048 bit long modulus (2 primes)
..............................................................+++++
..........................................................+++++
e is 65537 (0x010001)
name is expected to be in the format /type0=value0/type1=value1/type2=... where characters may be escaped by \. This name is not in that format: 'C:/Program Files/Git/CN=vpa_webhook_ca'
problems making Certificate Request
sh: __git_ps1: command not found

[bskiba]

This is a very standard script for generating webhook certificates for Kubernetes: https://github.com/kubernetes/apiserver/blob/master/pkg/util/webhook/gencerts.sh

The problem seems to be in this line: https://github.com/kubernetes/autoscaler/blob/master/vertical-pod-autoscaler/pkg/admission-controller/gencerts.sh#L40

Can you try to change it to:
openssl req -x509 -new -nodes -key ${TMP_DIR}/caKey.pem -days 100000 -out ${TMP_DIR}/caCert.pem -subj "/CN=vpa_webhook_ca"

[AdamSzendrei]

when I start the script vpa-up.sh I get the following error:

Generating certs for the VPA Admission Controller in /tmp/vpa-certs.
Generating RSA private key, 2048 bit long modulus
............+++
.......+++
e is 65537 (0x10001)
unknown option -addext
req [options]

[bskiba]

@AdamSzendrei please take a look at
https://github.com/kubernetes/autoscaler/tree/master/vertical-pod-autoscaler#install-command

[AdamSzendrei]

(1) my openssl is up-to-date and I still have the same issue.
(2) using vpa-release-0.8 solved the issue and I could pull the image

Thanks