Aks-engine: Requirements to customize the AKS Engine image

[firemanxbr]

Currently, we don't have in the README.md and in any another document our specifications about What are our requirements to customize the AKS Engine? (in terms of Image). We have some users with complaints in our Slack #sig-azure sharing bad experiences and issues when they tried to change the Ubuntu 16 LTS to Debian, CentOS or other Linux distributions. Will be nice if we can describe what we recommend as supported by this community, for example:

REQUIREMENTS

Supported Linux Distribution: Ubuntu 16 LTS
Packages:

• cgroups-lite
• curl
• etc...

This type of document can solve most of the questions and future issues related to the Linux Image.

[welcome[bot]]

👋 Thanks for opening your first issue here! If you're reporting a 🐞 bug, please make sure you include steps to reproduce it.

[firemanxbr]

This is not an issue reproducible but the gap of information that we have in the project and it's generating confusion with our users.

[jackfrancis]

Hi @firemanxbr,

There's no simple guidance at this point, as the project isn't evolved enough to the point where we have a nice, clean "plugin model" for adding distinct OS distros.

Here is a reference of an in-progress effort to enable Ubuntu 18.04-LTS:

https://github.com/Azure/aks-engine/pull/223

I think a practical first step would be identifying specific customizations that are desired, and to tackle those one-at-a time. Realistically, having a clean platform to drop in customizations is not likely to land anytime soon.

Thoughts?

[firemanxbr]

Hi @jackfrancis,

Thank you for share it. I believe with this information we can track the work in development to support Ubuntu 18.04-LTS to understand better this process and analyze if will possible send some PRs to the project or move our requirements to the Ubuntu distribution. I'll close the issue because of the most important to us was get the information about the process and we got! Thank you again.

[jackfrancis]

@CecileRobertMichon FYI: I think a good (non-critical) goal of the 18.04-LTS PR will be to make it a representative example of "how to add support for a new Azure VM SKU to AKS Engine". Let's hold ourselves to that standard as we craft the PR for merge:

• a single, canonical PR
• well decorated/described
• when appropriate, let's choose to decompose the 18.04-LTS implementation as distinct from 16.04-LTS to demonstrate how these SKUs can be implemented distinctly

[dcbrown16]

Just for reference, we received a Pro Direct support issue asking how to add CentOS using aks-engine, stemming perhaps from some unclear wording on this doc.
https://docs.microsoft.com/en-us/azure/aks/concepts-clusters-workloads
I'm communicating for now that it doesn't appear to be currently supported, but a list of what actually IS supported would be very nice.

[jackfrancis]

16.04-LTS is currently "supported", as in all functionality exists in there.

CoreOS-backed clusters can work, see:

https://github.com/Azure/aks-engine/pull/892
https://github.com/Azure/aks-engine/pull/1043 (this fixes a bug in #892)

And 18.04-LTS works, with the exception that Azure CNI support is not yet available:

https://github.com/Azure/aks-engine/pull/223