Crossplane: The key port and its value of connection secret didn't set to db-conn

[zzxwill]

What happened?

For alibaba provider PoC, followed publish infrastructure to consume infrastructure by creating a pod with command kubectl apply -f https://raw.githubusercontent.com/crossplane/crossplane/release-0.11/docs/snippets/publish/pod.yaml and it failed.

➜  /Users/zhouzhengxi/Downloads/crossplane k describe pod see-db-2
Name:         see-db-2
Namespace:    default
Priority:     0
Node:         cn-beijing.172.17.1.25/172.17.1.25
Start Time:   Sun, 24 May 2020 18:11:32 +0800
Labels:       <none>
Annotations:  kubernetes.io/psp: ack.privileged
Status:       Pending
IP:           192.168.0.220
IPs:
  IP:  192.168.0.220
Containers:
  see-db:
    Container ID:
    Image:         postgres:9.6
    Image ID:
    Port:          <none>
    Host Port:     <none>
    Command:
      psql
    Args:
      -c
      SELECT current_database();
    State:          Waiting
      Reason:       CreateContainerConfigError
    Ready:          False
    Restart Count:  0
    Environment:
      PGDATABASE:  postgres
      PGHOST:      <set to the key 'endpoint' in secret 'db-conn'>  Optional: false
      PGUSER:      <set to the key 'username' in secret 'db-conn'>  Optional: false
      PGPASSWORD:  <set to the key 'password' in secret 'db-conn'>  Optional: false
      PGPORT:      <set to the key 'port' in secret 'db-conn'>      Optional: false
    Mounts:
      /var/run/secrets/kubernetes.io/serviceaccount from default-token-2zwhz (ro)
Conditions:
  Type              Status
  Initialized       True
  Ready             False
  ContainersReady   False
  PodScheduled      True
Volumes:
  default-token-2zwhz:
    Type:        Secret (a volume populated by a Secret)
    SecretName:  default-token-2zwhz
    Optional:    false
QoS Class:       BestEffort
Node-Selectors:  <none>
Tolerations:     node.kubernetes.io/not-ready:NoExecute for 300s
                 node.kubernetes.io/unreachable:NoExecute for 300s
Events:
  Type     Reason     Age                From                             Message
  ----     ------     ----               ----                             -------
  Normal   Scheduled  <unknown>          default-scheduler                Successfully assigned default/see-db-2 to cn-beijing.172.17.1.25
  Normal   Pulled     65s (x2 over 66s)  kubelet, cn-beijing.172.17.1.25  Container image "postgres:9.6" already present on machine
  Warning  Failed     65s (x2 over 66s)  kubelet, cn-beijing.172.17.1.25  Error: couldn't find key port in Secret default/db-conn

How can we reproduce it?

Follow instructions per publish infrastructure

What environment did it happen in?

Crossplane version:

v 0.11.0

Cloud provider or hardware configuration

Kubernetes version (use kubectl version)

➜  /Users/zhouzhengxi/Downloads kubectl version
Client Version: version.Info{Major:"1", Minor:"18", GitVersion:"v1.18.2", GitCommit:"52c56ce7a8272c798dbc29846288d7cd9fbae032", GitTreeState:"clean", BuildDate:"2020-04-16T11:56:40Z", GoVersion:"go1.13.9", Compiler:"gc", Platform:"darwin/amd64"}
Server Version: version.Info{Major:"1", Minor:"16+", GitVersion:"v1.16.6-aliyun.1", GitCommit:"27fbcf7", GitTreeState:"", BuildDate:"2020-03-11T10:33:57Z", GoVersion:"go1.13.5", Compiler:"gc", Platform:"linux/amd64"}

• Kubernetes distribution (e.g. Tectonic, GKE, OpenShift)

Alibaba ACS

• OS (e.g. from /etc/os-release)
  System Version: macOS 10.15.3 (19D76)
• Kernel (e.g. uname -a)

Darwin zhouzhengxi.local 19.3.0 Darwin Kernel Version 19.3.0: Thu Jan  9 20:58:23 PST 2020; root:xnu-6153.81.5~1/RELEASE_X86_64 x86_64

[zzxwill]

I'm not sure whether it was caused by crossplane or provider-alibaba, reading those specs and going through source code:)

BTW, by hardcoding the value of port in pod.yaml, infrastructure could be successfully consumed.

apiVersion: v1
kind: Pod
metadata:
  name: see-db
  namespace: default
spec:
  containers:
  - name: see-db
    image: postgres:9.6
    command: ['psql']
    args: ['-c', 'SELECT current_database();']
    env:
    - name: PGDATABASE
      value: postgres
    - name: PGHOST
      valueFrom:
        secretKeyRef:
          name: db-conn
          key: endpoint
    - name: PGUSER
      valueFrom:
        secretKeyRef:
          name: db-conn
          key: username
    - name: PGPASSWORD
      valueFrom:
        secretKeyRef:
          name: db-conn
          key: password
    - name: PGPORT
      value: "3433"

[hasheddan]

@zzxwill thanks for opening this! I think I know what is happening here and will have a fix in shortly :) The issue is likely that the port key only ever gets written on the Create call, and not on any subsequent Observe. This means if it is not present on the object returned from CreateDBInstance then it will not ever be written in the ConnectionDetails, which dictate what appears on the Secret. I have seen this example work for me, so it seems that the port is not consistent as to whether it is returned on the Create call or not. Nevertheless, it should be included in the Observe as well so that the most up to data information is always being written.

[hasheddan]

This API spec does seem to indicate the the port should be present in the response from a CreateDBInstance request: https://www.alibabacloud.com/help/doc-detail/26228.htm

[hasheddan]

However, it does not appear that port is actually returned in the DescribeDBInstances call, which is being used in Observe, so simply adding it there will likely not fix the problem

[hasheddan]

@zzxwill after https://github.com/crossplane/provider-alibaba/pull/19 I am seeing the port value appearing consistently :) Feel free to try it out again from master and let us know if you have any trouble!

apiVersion: packages.crossplane.io/v1alpha1
kind: ClusterPackageInstall
metadata:
  name: provider-alibaba
  namespace: crossplane-system
spec:
  package: "crossplane/provider-alibaba:master"

[zzxwill]

Cool, It worked. Thanks for the quick fix.

➜  /Users/zhouzhengxi/Downloads/crossplane k describe secret db-conn
Name:         db-conn
Namespace:    default
Labels:       <none>
Annotations:  from.propagate.crossplane.io/name: 14f2df16-ea0b-4ee4-9428-40b813fc5dd3
              from.propagate.crossplane.io/namespace: crossplane-system

Type:  connection.crossplane.io/v1alpha1

Data
====
username:  6 bytes
endpoint:  40 bytes
password:  27 bytes
port:      4 bytes