Autoscaler: scale-down-delay-after-delete parameter doesn't work properly

[marwanad]

I believe that started around the time the switch to the new scale down processor happened.

The parameter goes into effect and puts scaledown in cooldown mode based on the lastScaleDownDeleteTime.

https://github.com/kubernetes/autoscaler/blob/774390efc80625466794131418b95092a5e63741/cluster-autoscaler/core/static_autoscaler.go#L499

However, this is only set when the scale down status result is ScaleDownNodeDeleted

https://github.com/kubernetes/autoscaler/blob/774390efc80625466794131418b95092a5e63741/cluster-autoscaler/core/static_autoscaler.go#L532-L535

And it seems with the switch to the async deletions, this is no longer set:

https://github.com/kubernetes/autoscaler/blob/30719059bf89199a983fde72c3e061561d280672/cluster-autoscaler/core/scale_down.go#L943-L964

/kind bug

[marwanad]

@MaciekPytel @towca

[marwanad]

Should we be returning ScaleDownNodeDeleted instead of ScaleDownNodeDeleteStarted - it seems that the later isn't used around so it seems like the most straightforward fix

[towca]

Thanks for pointing this out Marwan! Although I'd stick to using ScaleDownNodeDeleteStarted since it better conveys what actually happens. I'll send out a fix shortly.

[towca]

The bug should be fixed by #3570.

[ryaneorth]

thanks @towca @marwanad for finding/fixing this issue! any idea when/if this will make it into a patch release for versions 1.16.x/1.17.x?

[ryaneorth]

@MaciekPytel - maybe you know the answer to the above question?

[marwanad]

@ryaneorth we can prepare the cherry-pick PRs. I'm guessing we'll have one set of patch releases before K8s 1.20 and another around 1.20.

[ryaneorth]

Sounds great, thanks @marwanad . I'm happy to perform the cherry-picks if you'd like - let me know!

[marwanad]

@ryaneorth that would be great, thanks!

[ryaneorth]

Done!

3597

3598

3599

3600

[ryaneorth]

All of the above cherry-picks are complete. @MaciekPytel - do you have any information as to when the next patches will be released?

[marwanad]

@ryaneorth keep an 👀 on https://github.com/kubernetes/autoscaler/issues/3611