Azure-docs: AD login doesn't work after following instructions

[cocowalla]

I'm using a VM running Ubuntu Server 18.04, and I want to enable AAD login for SSH access. I've followed all the instructions for enabling SSH AAD login. Note that I had the same issue as #9418, and had to specify the UPN from az ad user list (which was in the form colin.anderson_mydomain.co.uk#EXT#@myotherdomain.onmicrosoft.com).

I can see in the portal that the extension is installed, and the roles granted. However, attempting to login with SSH just shows the usual password prompt (rather than prompting me to go to https://microsoft.com/devicelogin):

[email protected]@my-vm's password:

Note that if I login to the machine using public key auth, I'm able to su to the AAD user, but get a warning:

my-user@my-vm:~$ su [email protected]
To sign in, use a web browser to open the page https://microsoft.com/devicelogin and enter the code ABCDEFGHI to authenticate.
Failed to set some of the admin privileges. Running with decreased permissions.

Here is some possible relevant stuff from /var/log/auth.log from when I su'd to the AAD user:

Dec  7 10:08:31 my-vm su[11705]: pam_aad(su:auth): Version: 1.0.006350001; CorrelationId: b7ffd7c0-a8f2-4f1b-88d9-4ab23f2a0844
Dec  7 10:08:47 my-vm su[11705]: pam_aad(su:auth): Login granted for [email protected] as an admin.
Dec  7 10:08:47 my-vm su[11705]: pam_aad(su:auth): Adding [email protected] to aad_admins.
Dec  7 10:08:47 my-vm gpasswd[12173]: my-vm failed to add user [email protected] to group aad_admins: Permission denied
Dec  7 10:08:47 my-vm su[11705]: pam_aad(su:auth): gpasswd: Permission denied.
Dec  7 10:08:47 my-vm su[11705]: pam_aad(su:auth): User membership change failed with error code 256
Dec  7 10:08:47 my-vm su[11705]: pam_aad(su:auth): Adding [email protected] to admin.
Dec  7 10:08:47 my-vm gpasswd[12174]: my-vm failed to add user [email protected] to group admin: Permission denied
Dec  7 10:08:47 my-vm su[11705]: pam_aad(su:auth): gpasswd: Permission denied.
Dec  7 10:08:47 my-vm su[11705]: pam_aad(su:auth): User membership change failed with error code 256
Dec  7 10:08:47 my-vm su[11705]: Successful su for [email protected] by my-vm

I've tried rebooting, and also stopping/starting the VM, but the problem persists.

---

Document Details

⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

• ID: bb8e2623-1f79-aee3-bd97-8b5f647b6980
• Version Independent ID: 6346b54d-cc8d-0b79-a6b8-f9144cfa2459
• Content: Log in to a Linux VM with Azure Active Directory credentials
• Content Source: articles/virtual-machines/linux/login-using-aad.md
• Service: virtual-machines-linux
• GitHub Login: @cynthn
• Microsoft Alias: cynthn

[cocowalla]

So, I got this working after coming across this issue.

I think it would be good to update the docs to say that /etc/ssh/sshd_config needs the following set:

PasswordAuthentication no
ChallengeResponseAuthentication yes

[mimckitt]

Thanks for the feedback! We are currently investigating and will update you shortly.

[yyanev]

The problem is this #EXT# thing. The UPN check doesn't recognize # and upper case characters. We should add the # character. However you can use your e-mail instead of UPN.
The new version will have a better message in case you use upper case.

[DLN-India]

Unable to ssh into a newly created VM, reasons same as the error above. It just goes to an infinite loop of entering code at devicelogin

[Karishma-Tiwari-MSFT]

@DLN-India Did you get a chance to try the solution that solved the issue for the original poster? Let me know.

[klchung]

Executed the instructions (preview) provided and also updated the sshd_config and aad_admins, but the same error persists, i.e. "Permission denied" when attempting to ssh login with my AAD account to a SLES 12 SP3 Linux virtual machine.

Please advise on next steps, and thanks in advance.

[klchung]

Got it working with additional steps below after following the instructions provided:

1. verify home directory of AAD account in SLES VM via: getent
2. create home directory, e.g. /home/
3. create the .ssh subdirectory
4. create the "authorized_keys" and populate with public keys for remote access without password prompt

[aavdberg]

Getting the following error:

he authenticity of host 'xx.xxx.xxx.xxx (xx.xxx.xxx.xxx)' can't be established.
ECDSA key fingerprint is SHA256:BA+bC05bkMZNBRWUxL11Lu/whfLAi/oozQaKZZx5nHA.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'xx.xxx.xxx.xxx' (ECDSA) to the list of known hosts.
[email protected]@xx.xxx.xxx.xxx: Permission denied (publickey,keyboard-interactive).

When I did it Friday it worked, but sunday it did not work anymore.

[yyanev]

Can you provide some more information like the Linux distro and version? Are you trying to login with the device flow or with a SSH key?
The login with a SSH key is no longer allowed for AAD users. Otherwise someone can login, install a key and after that will be able to bypass all RBAC checks.

From: Andre van den Berg [e-mail]
Sent: Monday, April 15, 2019 11:33 AM
To: MicrosoftDocs/azure-docs [e-mail]
Cc: Yancho Yanev [e-mail]; Comment [e-mail]
Subject: Re: [MicrosoftDocs/azure-docs] AD login doesn't work after following instructions (#20171)

Getting the following error:

he authenticity of host 'xx.xxx.xxx.xxx (xx.xxx.xxx.xxx)' can't be established.
ECDSA key fingerprint is SHA256:BA+bC05bkMZNBRWUxL11Lu/whfLAi/oozQaKZZx5nHA.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'xx.xxx.xxx.xxx' (ECDSA) to the list of known hosts.
[email protected]@[e-mail]: Permission denied (publickey,keyboard-interactive).

When I did it Friday it worked, but sunday it did not work anymore.

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHubhttps://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FMicrosoftDocs%2Fazure-docs%2Fissues%2F20171%23issuecomment-483367096&data=02%7C01%7Cyyanev%40microsoft.com%7C95d5cc155afb4ee2773908d6c1d0c641%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636909499757651744&sdata=YuB757kutbtqCajrrtBiSiij92C45r%2B92Gsu87cG%2FyI%3D&reserved=0, or mute the threadhttps://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAedLvIig07qFt7Thph7VdjXD8xAkG28Mks5vhMXWgaJpZM4ZILjc&data=02%7C01%7Cyyanev%40microsoft.com%7C95d5cc155afb4ee2773908d6c1d0c641%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636909499757656735&sdata=lYK6oXf5nt%2FzRrdr2M4fNYTC6K1fgKnqJVPgZrO%2F%2Fkc%3D&reserved=0.

[AlexBevan]

Just to echo @aavdberg we are having the same issue, used last week without issue but not today, so looks like it may be more widely spread.

[Karishma-Tiwari-MSFT]

@yyanev Is it a known issue? Please share your insights.
@aavdberg Can you please provide the details as asked above so that our team can help you better.

[AlexBevan]

Virtual machine name
alexgithubissue
Region
UK South
Availability options
No infrastructure redundancy required
Authentication type
Password
Username
azureuser
Public inbound ports
SSH
Login with Azure Active Directory (Preview)
On
"imageReference": {
"publisher": "Canonical",
"offer": "UbuntuServer",
"sku": "18.04-LTS",
"version": "latest"
}

ARM Template here: https://alexbevan.uk/template-gh-issue.zip

[Karishma-Tiwari-MSFT]

Thanks @AlexBevan for sharing the details.
The product team are made aware of this issue and they are investigating. I will share any updates here.

[lwillek]

Maybe caused by package "aadlogin_1.0.008300001_amd64.deb" released on 12-Apr-2019 19:33
No problem here with older deb package aadlogin_1.0.006350001_amd64.deb.
OS: latest Ubuntu 18.04.2 LTS (codename: bionic) with all updates installed.
MS case 119041621001325

[yyanev]

The new version doesn’t allow public key authentication for AAD users, which bypasses the RBAC checks. If you have an installed public key, follow these steps:

1. Log on with disabled public key (ssh -o PubkeyAuthentication=no example.com)
2. Delete any public keys stored on the VM
3. Now you can log in without the extra flag

From: Lutz Willek [e-mail]
Sent: Wednesday, April 17, 2019 12:29 PM
To: MicrosoftDocs/azure-docs [e-mail]
Cc: Yancho Yanev [e-mail]; Mention [e-mail]
Subject: Re: [MicrosoftDocs/azure-docs] AD login doesn't work after following instructions (#20171)

Maybe caused by package "aadlogin_1.0.008300001_amd64.deb" released on 12-Apr-2019 19:33
No problem here with older deb package aadlogin_1.0.006350001_amd64.deb.
OS: latest Ubuntu 18.04.2 LTS (codename: bionic) with all updates installed.
MS case 119041621001325

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHubhttps://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FMicrosoftDocs%2Fazure-docs%2Fissues%2F20171%23issuecomment-484228705&data=02%7C01%7Cyyanev%40microsoft.com%7C7528392f978b4200c2d808d6c36af3fd%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636911261455320293&sdata=9w9aCJdnyj8iCNxTqQcUUEhrrvbO3GqH0x%2BaXVHqnb4%3D&reserved=0, or mute the threadhttps://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAedLvBzqyJjn6xbaZq6UJAO18IgEWbV5ks5vh3YAgaJpZM4ZILjc&data=02%7C01%7Cyyanev%40microsoft.com%7C7528392f978b4200c2d808d6c36af3fd%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636911261455330287&sdata=Odpbn4j4XqO%2Bq0KDEaqjdM5wH4%2FE5hjpctUIzqwnAq4%3D&reserved=0.

[yyanev]

Adding Sandeep, our PM.

If we don’t have this in the documentation, we should add it.

In the case when we discover an AAD user trying to log in with a public key, we try to send this message to the user:
AAD users are not allowed to use public key authentication. Please add '-o PubkeyAuthentication=no' to your ssh command and try again. You may also want to remove .ssh/authorized_keys file to prevent future login failures.

However some clients don’t show the callback messages if the login is not successful.

From: Yancho Yanev
Sent: Wednesday, April 17, 2019 12:51 PM
To: MicrosoftDocs/azure-docs [e-mail]; MicrosoftDocs/azure-docs [e-mail]
Cc: Mention [e-mail]
Subject: RE: [MicrosoftDocs/azure-docs] AD login doesn't work after following instructions (#20171)

The new version doesn’t allow public key authentication for AAD users, which bypasses the RBAC checks. If you have an installed public key, follow these steps:

1. Log on with disabled public key (ssh -o PubkeyAuthentication=no example.com)
2. Delete any public keys stored on the VM
3. Now you can log in without the extra flag

From: Lutz Willek <[email protected][e-mail]>
Sent: Wednesday, April 17, 2019 12:29 PM
To: MicrosoftDocs/azure-docs <[email protected][e-mail]>
Cc: Yancho Yanev <[email protected][e-mail]>; Mention <[email protected][e-mail]>
Subject: Re: [MicrosoftDocs/azure-docs] AD login doesn't work after following instructions (#20171)

Maybe caused by package "aadlogin_1.0.008300001_amd64.deb" released on 12-Apr-2019 19:33
No problem here with older deb package aadlogin_1.0.006350001_amd64.deb.
OS: latest Ubuntu 18.04.2 LTS (codename: bionic) with all updates installed.
MS case 119041621001325

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHubhttps://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FMicrosoftDocs%2Fazure-docs%2Fissues%2F20171%23issuecomment-484228705&data=02%7C01%7Cyyanev%40microsoft.com%7C7528392f978b4200c2d808d6c36af3fd%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636911261455320293&sdata=9w9aCJdnyj8iCNxTqQcUUEhrrvbO3GqH0x%2BaXVHqnb4%3D&reserved=0, or mute the threadhttps://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAedLvBzqyJjn6xbaZq6UJAO18IgEWbV5ks5vh3YAgaJpZM4ZILjc&data=02%7C01%7Cyyanev%40microsoft.com%7C7528392f978b4200c2d808d6c36af3fd%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636911261455330287&sdata=Odpbn4j4XqO%2Bq0KDEaqjdM5wH4%2FE5hjpctUIzqwnAq4%3D&reserved=0.

[richardbolt]

Is there any way to disable the newer behavior of disallowing ssh key login?
We preferred the original way of being able to setup an ssh key after an initial login.

Not being able to do this means our use case as a jump box to connect to Azure managed MySQL instances via an SSH tunnel doesn't work and we end up with less security as we have to whitelist on the DB firewalls our office IPs. The original way at least users had to setup an ssh key on the jump box first which in turn meant they had to be in a certain AD group with the correct role assigned to do it. By disallowing the option to disable the no-ssh key option we have less data security overall than before.

[cynthn]

reassign @SanDeo-MSFT

[yyanev]

This is the way the AAD login currently works:

1. The PAM module executes a device code flow. At the end we have the user access token. The token is locked down to be able only to call CheckMyAccess endpoint.
2. The PAM module calls CheckMyAccess to see if the user has rights to access this VM and also adjusts the membership to the aad_admins group.

If the user is allowed to install keys, the subsequent requests will bypass the device code flow. Then there is no way to get the user access token and therefore no way to check if the user still has access to this VM. This is the reason why they keys are disallowed.

SSH keys still work for local accounts. The trick is to make sure the local user name doesn’t looks like an email.

We know the pain caused by the requirement of VMs to have access to some Azure endpoints and are looking for ways to solve this problem. At some point we will either automatically whitelist the required endpoints during installation or will tunnel the calls through the metadata service on the host (http://169.254.169.254/metadata). For not though we don’t have this piece of the puzzle.

From: Richard Bolt [e-mail]
Sent: Wednesday, July 24, 2019 9:42 AM
To: MicrosoftDocs/azure-docs [e-mail]
Cc: Yancho Yanev [e-mail]; Mention [e-mail]
Subject: Re: [MicrosoftDocs/azure-docs] AD login doesn't work after following instructions (#20171)

Is there any way to disable the newer behavior of disallowing ssh key login?
We preferred the original way of being able to setup an ssh key after an initial login.

Not being able to do this means our use case as a jump box to connect to Azure managed MySQL instances via an SSH tunnel doesn't work and we end up with less security as we have to whitelist on the DB firewalls our office IPs. The original way at least users had to setup an ssh key on the jump box first which in turn meant they had to be in a certain AD group with the correct role assigned to do it. By disallowing the option to disable the no-ssh key option we have less data security overall than before.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHubhttps://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FMicrosoftDocs%2Fazure-docs%2Fissues%2F20171%3Femail_source%3Dnotifications%26email_token%3DAHTUXPES3PIEM2LUIRW3JZDQBCA4RA5CNFSM4GJAXDOKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD2W5LVA%23issuecomment-514708948&data=02%7C01%7Cyyanev%40microsoft.com%7Ce26f247cea9c4c8accbf08d71055d0fd%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636995833076164852&sdata=da7zQiSx%2BG2Wfrxrt9%2BNZqqD%2FHgal7liqEBYqjTS10o%3D&reserved=0, or mute the threadhttps://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAHTUXPEUIAS426NJBBE4EB3QBCA4RANCNFSM4GJAXDOA&data=02%7C01%7Cyyanev%40microsoft.com%7Ce26f247cea9c4c8accbf08d71055d0fd%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636995833076164852&sdata=V7OL9DxcBcbAgLKiDIgbezlG%2F2mQaec%2BrUGyVmtKm70%3D&reserved=0.

[mimckitt]

We are currently closing out long standing backlog in order to better determine which issues are still valid and which have been addressed. If you are still seeing a specific problem with this document please open a new issue so we can properly assign the feedback to the correct author and get it addressed ASAP.

please-close

[pierreyvesv]

sorry to comment in a closed ticket but I just found a work around:
in the file /etc/pam.d/common-account comment out the line:
account [success=2 ignore=ignore default=die] pam_aad.so

then you'll be able to authenticate to the server with your ssh key