Aks-engine: Password expired when not using a password

[davidmatson]

Describe the bug
Logging into an aks-engine master deployed a while ago, it reports that the password must be changed, but we're using only SSH keys, not a password, so we don't know the current password and thus can no longer log in.

Steps To Reproduce
Use an apimodel with data such as:

    "linuxProfile": {
      "adminUsername": "<username>",
      "ssh": {
        "publicKeys": [
          {
            "keyData": "ssh-rsa <base64> <username>@<hostname>"
          }
        ]
      }

Expected behavior
There is no password, or at least it doesn't expire.

AKS Engine version
0.38.2

Kubernetes version
1.15.0

Additional context
Here's the console output I get logging in with the SSH key:

WARNING: Your password has expired.
You must change your password now and login again!
Changing password for <username>.
(current) UNIX password:

I do get this error for clusters deployed on the following dates:
2019-07-29
2019-07-16
2019-06-12

but not for a cluster deployed on:
2019-09-15

so the password appears to be expiring sometime between 52 and 100 days after cluster deployment.

[CecileRobertMichon]

Have you tried resetting the password in the portal or with the Azure CLI to a new valid SSH key?

[davidmatson]

I went to the portal, but it says this (I'm using VMSS masters):

Update the admin credentials directly in the scale set model. Once the scale set is updated, all new VMs have the new credentials. Existing VMs only have the new credentials if they are updated to the latest model and then reimaged.

I'd rather not have to reimage all masters periodically due to password expiration. Do you know what the root cause for the password expiration? Is it a text password expiring or an SSH key expiring? (I didn't think of SSH keys as expiring, but perhaps that's possible.)

[devigned]

If we are not already, we should be disabling password expiration when passwords are disabled. https://superuser.com/questions/576932/turning-off-password-expiration-on-linux

As for the current situation, I believe the above link will help you mitigate the issue.

[CecileRobertMichon]

@devigned @jackfrancis pretty sure we had to enable password expiration for CIS benchmark compliance https://github.com/Azure/aks-engine/pull/1162

[davidmatson]

Is password expiration required for CIS benchmark compliance even when not using a password?

[devigned]

If passwords are disabled, then expiration seems superfluous. Am I missing something?

[CecileRobertMichon]

I agree. Right now the expiration enforcement is definitely not "smart" about it, it just enables it at the OS image level (in the prebaked VHD). I'm not sure if the benchmark itself is smarter and that it only fails that check if passwords are enabled.

[jackfrancis]

Probably: CIS is missing something, and certain users trust CIS errors as static truth.

This sounds like something we should clarify and clean up.

[panicstevenson]

We recently ran into this issue as well -- it looks like the change was added via https://github.com/Azure/aks-engine/pull/1162, so this has been around since April. The default account expiration is 90 days and accounts without passwords will expire just the same.

We used the following commands to work around the issue:
VM:

az vm run-command invoke -g <resource-group> -n <virtual-machine-name> --command-id RunShellScript --scripts "sudo chage -I -1 -m 0 -M 99999 -E -1 <your-username-here>"

VMSS:

az vmss run-command invoke -g <resource-group> -n <scale-set-name> --instance-id 1 --command-id RunShellScript --scripts "sudo chage -I -1 -m 0 -M 99999 -E -1 <your-username-here>"

Obviously all this is doing is setting the account to never expire, but that was sufficient in our case.

[jackfrancis]

Thank you so much for sharing your remediation steps @panicstevenson!