Aks-engine: Addons are not started for K8s > 1.16-alpha.0.

[adelina-t]

K8s 1.16 is deprecating some beta workload APIs [1] , thus all common resources like DaemonSets, Deployments, ReplicaSets will be moved from extensions/v1beta1 to apps/v1 by default and the extensions/v1beta1endpoints will not be enabled by default anymore ( See linked PR for all resources changed ).

Since most of the manifests for the addons used by aks-engine use v1beta1 api version ( example [2] ), these resources will not be able to start when using k8s versions newer than 1.16-alpha.0.

We noticed this issue in the upstream Windows K8s CI [3]. For anyone that might be affected by this ( sig-azure CI ? https://testgrid.k8s.io/sig-azure-master ), the quick workaround is to just reenable the endpoints in the api-server by passing --runtime-config flag. The longterm option is of course to change the manifests for the addons.

[1] https://github.com/kubernetes/kubernetes/pull/70672
[2] https://github.com/Azure/aks-engine/blob/master/parts/k8s/addons/kubernetesmasteraddons-kube-proxy-daemonset.yaml
[3] https://testgrid.k8s.io/sig-windows#aks-engine-azure-windows-master-staging

[adelina-t]

@CecileRobertMichon @jackfrancis Keep this in mind for when you plan to start supporting alpha versions of 1.16 :)

[adelina-t]

cc @PatrickLang

[CecileRobertMichon]

Thanks @adelina-t

also @mboersma FYI

[jackfrancis]

We'll change the v1beta1 to apps/v1 pro-actively, if that evolution works with < 1.16 versions then we can do this pretty quickly.

[adelina-t]

I think apps/v1 works down to k8s 1.14

Sent from my iPhone

On 24 Jun 2019, at 20:07, Jack Francis <[email protected][e-mail]> wrote:

We'll change the v1beta1 to apps/v1 pro-actively, if that evolution works with < 1.16 versions then we can do this pretty quickly.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHubhttps://github.com/Azure/aks-engine/issues/1522?email_source=notifications&email_token=ABX3CFXZZTCU3SZK3EFKNKTP4D5NBA5CNFSM4H27Z7M2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODYNSW5Y#issuecomment-505097079, or mute the threadhttps://github.com/notifications/unsubscribe-auth/ABX3CFQYGX3QSSPF36SNH63P4D5NBANCNFSM4H27Z7MQ.