Cluster-api-provider-azure: move to clusterctl is a regression

[dklyle]

/kind bug

What steps did you take and what happened:
The move to use clusterctl instead of templates in the cluster-api-provider-azure directory is a regression. clusterctl is only nice for generating the default cluster everyone else is generating. Once you advance to wanting to modify your cluster configuration to deviate from the default, the path becomes rather obscure. The way to override the templates is using a dot directory in the user's home directory, .cluster-api. This is not well documented at all. This also method is not a realistic deployment expectation. The older approach of using kustomize on the template files is much more straightforward. Noting that the Makefile in cluster-api-provider-azure also uses this approach for development. Magic is nice when you're experimenting to spin up your first cluster, but it makes for difficulties when customizing for production deployments.

What did you expect to happen:
The older approach of using kustomize on the template files is much more straightforward.

Environment:

• cluster-api-provider-azure version: 0.4.3
• Kubernetes version: (use kubectl version): 1.17.4
• OS (e.g. from /etc/os-release): Ubuntu 18.04.4

[CecileRobertMichon]

cc @fabriziopandini @vincepri FYI

Thanks for the feedback @dklyle

[vincepri]

Thanks for the feedback @dklyle, this has been something that has been on our minds as well. Clusterctl has been redesigned during v1alpha3 to accomplish a few things: ease of use for newcomers, increase adoption, setup requirements, and in general having a nicer getting started experience.

In addition, clusterctl has also been made a library to allow for users to rely on well defined patterns that we'd expect them to take when spinning up a cluster.

Everything I've mentioned above, covers part of the use cases. More advanced users will find the current experience a little limiting. We've needed this and similar feedback for a while, if we can have some of your time, it'd be great to hear more about what you'd expect and how we can improve things.

There is _some_ documentation about how to use .cluster-api directory and some more advanced use cases in https://cluster-api.sigs.k8s.io/clusterctl/configuration.html and https://cluster-api.sigs.k8s.io/clusterctl/commands/commands.html.

Something to note though is that you don't necessarily have to use clusterctl, you can use parts of it (via a library) to get bootstrapped, and continue with the raw yaml files.

[fabriziopandini]

Thanks for the feedback!
I would only add that the work for enabling the usage of a templating engine in clusterctl creates the foundation for a middle ground between the initial implementation, oriented to simplicity/day0 experience, and the flexibility required by advanced users.
The next step is to get a first templating solution integrated in clusterctl, and for this, it will be really valuable to have some more details about @dklyle use cases.

[dklyle]

I appreciate the desire to move forward and create a better user experience for first time users. I have been using various cluster-api-providers for the past year, and the recent update to all documentation referencing only clusterctl led me to believe that was the only supported path. After making the overrides work for the most part, I did dig into the Makefile and realize dev work still relies on templates and kustomize and returned to using that method as it was much easier to have greater control.

I am glad a templating engine is being proposed. Please consider supporting the templating for config as well as using pre-downloaded versions of the various yaml file dependencies so that a deployer can pin to specific versions and not have to download them at cluster bring up.

Since the templating engine seems to be a bit in the future, I would recommend adding an advanced section in the documentation about using kustomize and templates in the meantime possibly with a note about the future templating engine in clusterctl once it is approved.

All that said, I really appreciate all the great work on cluster-api and the specific providers. It's a great tool and I look forward to future improvements.

[CecileRobertMichon]

Proposal in progress here: https://docs.google.com/document/d/1ZsusF5c9pYxseuaKxTpctI5aUDqzl0sdCW4xxDbLm3k

[fejta-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

[fejta-bot]

Stale issues rot after 30d of inactivity.
Mark the issue as fresh with /remove-lifecycle rotten.
Rotten issues close after an additional 30d of inactivity.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle rotten

[fejta-bot]

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen.
Mark the issue as fresh with /remove-lifecycle rotten.

Send feedback to sig-contributor-experience at kubernetes/community.
/close

[k8s-ci-robot]

@fejta-bot: Closing this issue.

In response to this:

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen.
Mark the issue as fresh with /remove-lifecycle rotten.

Send feedback to sig-contributor-experience at kubernetes/community.
/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[CecileRobertMichon]

https://github.com/kubernetes-sigs/cluster-api/pull/4333/files