Cluster-api: [clusterctl] "clusterctl config provider" fails to show AWS, VSphere, and Azure info

[sedefsavas]

What steps did you take and what happened:

Running clusterctl config provider --infrastructure aws gives the following error:

Error: failed to perform variable substitution: value for variables [AWS_B64ENCODED_CREDENTIALS] is not set. Please set the value using os environment variables or the clusterctl config file

What did you expect to happen:
I expected it to run without needing any variables similar to other providers.

/kind bug
/area clusterctl

[vincepri]

/close

We expect variables to be present to generate configuration.

@randomvariable mentioned during today's meeting that need for credentials might be removed.

[k8s-ci-robot]

@vincepri: Closing this issue.

In response to this:

/close

We expect variables to be present to generate configuration.

@randomvariable mentioned during today's meeting that need for credentials might be removed.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[CecileRobertMichon]

@vincepri @fabriziopandini thinking about this more, custerctl config provider is supposed to list info about a provider. 
It doesn’t generate a config or deploy any infrastructure, so not sure why it needs credentials...

FWIW the same is true with Azure:

clusterctl config provider --infrastructure azure                                                           
Error: failed to perform variable substitution: value for variables [AZURE_CLIENT_ID_B64, AZURE_CLIENT_SECRET_B64, AZURE_SUBSCRIPTION_ID_B64, AZURE_TENANT_ID_B64] is not set. Please set the value using os environment variables or the clusterctl config file

After exporting the credentials:

clusterctl config provider --infrastructure azure                                                                ✔  10270  10:31:28
Name:               azure
Type:               InfrastructureProvider
URL:                https://github.com/kubernetes-sigs/cluster-api-provider-azure/releases/latest/infrastructure-components.yaml
Version:            v0.4.1
TargetNamespace:    capz-system
WatchingNamespace:
Variables:
  - AZURE_CLIENT_ID_B64
  - AZURE_CLIENT_SECRET_B64
  - AZURE_SUBSCRIPTION_ID_B64
  - AZURE_TENANT_ID_B64
Images:
  - gcr.io/kubebuilder/kube-rbac-proxy:v0.4.1
  - us.gcr.io/k8s-artifacts-prod/cluster-api-azure/cluster-api-azure-controller:v0.4.1
  - gcr.io/kubebuilder/kube-rbac-proxy:v0.4.1
  - us.gcr.io/k8s-artifacts-prod/cluster-api-azure/cluster-api-azure-controller:v0.4.1

[CecileRobertMichon]

Also this works which proves the command doesn't need credentials to work:

export AWS_B64ENCODED_CREDENTIALS=fakecreds                                                          ]
 clusterctl config provider --infrastructure aws                                                               
Name:               aws
Type:               InfrastructureProvider
URL:                https://github.com/kubernetes-sigs/cluster-api-provider-aws/releases/latest/infrastructure-components.yaml
Version:            v0.5.2
TargetNamespace:    capa-system
WatchingNamespace:
Variables:
  - AWS_B64ENCODED_CREDENTIALS
Images:
  - gcr.io/kubebuilder/kube-rbac-proxy:v0.4.1
  - us.gcr.io/k8s-artifacts-prod/cluster-api-aws/cluster-api-aws-controller:v0.5.2
  - gcr.io/kubebuilder/kube-rbac-proxy:v0.4.1
  - us.gcr.io/k8s-artifacts-prod/cluster-api-aws/cluster-api-aws-controller:v0.5.2

[sedefsavas]

/reopen

[k8s-ci-robot]

@sedefsavas: Reopened this issue.

In response to this:

/reopen

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[nader-ziada]

/assign

[vincepri]

/milestone v0.3.x

[fabriziopandini]

custerctl config provider is supposed to list info about a provider. 
It doesn’t generate a config or deploy any infrastructure, so not sure why it needs credentials...

Two comments about this:

1. It requires all variables to be set, not only credentials
2. custerctl config provider is the -o yaml options that is the only way for a user to get a preview of the yaml that will be applied to a cluster

If we are going to make the command to work without variable substitution, we are losing the option to run o get a preview of the yaml that will be applied to a cluster, which can be useful.

I see three options:

• Have a different behavior between custerctl config provider (not requiring variables) and custerctl config provider -o yaml (requiring variables)
• Go for a middle-ground solution: if variables are available use them, but tolerate missing variables
• Drop the possibility to have a preview of the yaml that will be applied to a cluster

I'm for the middle-ground solution option. opinions?

[CecileRobertMichon]

If we are going to make the command to work without variable substitution, we are losing the option to run o get a preview of the yaml that will be applied to a cluster, which can be useful.

Isn't that what clusterctl config cluster does?

clusterctl config                                                         ✔  10738  16:50:25
Display provider configuration and templates to create workload clusters.

Usage:
  clusterctl config [command]

Available Commands:
  cluster      Generate templates for creating workload clusters.
  provider     Display information about a provider.
  repositories Display the list of providers and their repository configurations.

If is the only way for a user to get a preview of the yaml that will be applied to a cluster is true then maybe that means we are overloading the clusterctl config provider command to start with... The command is documented as Display information about a provider.

[fabriziopandini]

@CecileRobertMichon clusterctl config cluster gives you the cluster template yaml, while clusterctl config provider gives you the component yaml (your provider yaml)

[CecileRobertMichon]

edit: I realized that by "a preview of the yaml that will be applied to a cluster" you meant the infra components, not the cluster resources.

In that case I think solution 2) makes sense - the command shouldn't fail if variables are missing since it still provides valuable information and should just replace the variables if they are present.