Azure-sdk-for-js: Managed Identity Login Fails on local machine

[PIC123]

• Package Name: @azure/identity
• Package Version: 1.1.0
• Operating system: Windows 10
• [ ] nodejs
  
  
  
  • version:
  
  
• [ ] browser
  
  
  
  • name/version:
  
  
• [x] typescript
  
  
  
  • version: 4.0.2
  
  
• Is the bug related to documentation in
  
  
  
  • [ ] README.md
  
  
  • [ ] source code documentation
  
  
  • [ ] SDK API docs on https://docs.microsoft.com
  
  

Describe the bug
When doing local development with the azure/identity package, the package seems to still try and log in using managed identity and fails, even though managed identity shouldn't be used when doing local development and it should default to the Azure Account extension

To Reproduce
Steps to reproduce the behavior:

1. Create app that uses the DefaultAzureCrednetial
2. try to use the credential for anything, i.e. getting a token

Expected behavior
The package should fall back to using the Azure Account extension if managed identity login fails because of local dev

Screenshots

Additional context
I am an ms internal engineer, feel free to reach out for more context/clarifications

[mhoeger]

+1, seeing this issue intermittently :( also an ms internal engineer and can provide repro

[sadasant]

Hello @PIC123 and @mhoeger ! I'm Daniel,

I'll be doing my best to help you.

I'll have an answer for you shortly (as in, up to a couple of hours). Thank you for your patience!

[sadasant]

I am fairly sure that this is a misalignment we have between JS and other languages. The misalignment being that we should skip this error instead of throwing like that.

There's an issue to align with other languages already: https://github.com/Azure/azure-sdk-for-js/issues/9383#issuecomment-696915350

This specific version of that error shows that the credential tried to reach 169.254.169.254:80. I wonder where that comes from.

Let's go through what is currently happening:

• 169.254.169.254 is the ImdsEndpoint here.
• The first time this is executed, isEndpointUnavailable is null.
  
  
  
  • This means that checkIfImdsEndpointAvailable is true.
  
  
  • Which means pingImdsEndpoint is executed.
  
  
• So, first ping to this endpoint, just in case it's reachable (through the pingImdsEndpoint function).
  
  
  
  • This is wrapped in a try catch, and either returns true or false.
  
  
  • My expectation is that this is not the source of the error.
  
  
• Only if this is reachable, we attempt to authenticate against it.
• If the request fails, we currently throw with a AuthenticationError. We will throw with a CredentialUnavailable error after #9383 is addressed.

Now, when would the endpoint be available for ping but not for authentication? I'm guessing this would only happen if you're on the VPN.

Action Item:
Do you mind trying to ping 169.254.169.254? what happens when you do so?

If you're able to see this endpoint from your network, then I would say that my intuition is correct, and thus the fix for #9383 should resolve this issue.

I'll be asking around to see if somebody else has a better guess.

[sadasant]

I'm taking tomorrow off, but I'll come back to this on Monday for sure.

[mhoeger]

Hi @sadasant , thanks for your follow-up!! Unfortunately in my case, I'm not on VPN (@PIC123 if you are, I can open a separate ticket to track my issue)

When I tried pinging the endpoint, I got this:

Request timeout for icmp_seq 162
ping: sendto: Host is down
Request timeout for icmp_seq 163
ping: sendto: Host is down
Request timeout for icmp_seq 164
^C
--- 169.254.169.254 ping statistics ---
166 packets transmitted, 0 packets received, 100.0% packet loss

My exception details (again - can file a separate issue if helpful!)

[9/25/20 5:57:16 AM] Exception: AuthenticationError: ManagedIdentityCredential authentication failed. EHOSTDOWN_null_{"message":"request to http://169.254.169.254/metadata/identity/oauth2/token?resource=https%3A%2F%2Fmanagement.core.windows.net%2F&api-version=2018-02-01 failed, reason: connect EHOSTDOWN 169.254.169.254:80 - Local (192.168.1.161:49302)","type":"system","errno":"EHOSTDOWN","code":"EHOSTDOWN"}_(status code 400).
[9/25/20 5:57:16 AM] More details:
[9/25/20 5:57:16 AM] request to http://169.254.169.254/metadata/identity/oauth2/token?resource=https%3A%2F%2Fmanagement.core.windows.net%2F&api-version=2018-02-01 failed, reason: connect EHOSTDOWN 169.254.169.254:80 - Local (192.168.1.161:49302)
[9/25/20 5:57:16 AM] Stack: AuthenticationError: ManagedIdentityCredential authentication failed. EHOSTDOWN_null_{"message":"request to http://169.254.169.254/metadata/identity/oauth2/token?resource=https%3A%2F%2Fmanagement.core.windows.net%2F&api-version=2018-02-01 failed, reason: connect EHOSTDOWN 169.254.169.254:80 - Local (192.168.1.161:49302)","type":"system","errno":"EHOSTDOWN","code":"EHOSTDOWN"}_(status code 400).
[9/25/20 5:57:16 AM] More details:
[9/25/20 5:57:16 AM] request to http://169.254.169.254/metadata/identity/oauth2/token?resource=https%3A%2F%2Fmanagement.core.windows.net%2F&api-version=2018-02-01 failed, reason: connect EHOSTDOWN 169.254.169.254:80 - Local (192.168.1.161:49302)
[9/25/20 5:57:16 AM]     at ManagedIdentityCredential.<anonymous> (/Users/mariehoeger/Documents/repos/Connectors/samples/azurefunctions/memeGenerator/node_modules/@azure/identity/dist/index.js:1077:23)
[9/25/20 5:57:16 AM]     at Generator.throw (<anonymous>)
[9/25/20 5:57:16 AM]     at rejected (/Users/mariehoeger/Documents/repos/Connectors/samples/azurefunctions/memeGenerator/node_modules/@azure/identity/node_modules/tslib/tslib.js:112:69)
[9/25/20 5:57:16 AM]     at process._tickCallback (internal/process/next_tick.js:68:7).

[PIC123]

I'm also not on a vpn, but a thing to note if forgot to mention earlier, the azure sub I'm trying to hit is on a private proxy so that may cause some issues too

[sadasant]

I got in contact with @mhoeger and I was able to reproduce this. I'm actively working on it.

[sadasant]

I was able to test it and find a fix. I've added this fix to an existing pull request. Therefore: This fix will go out next week. Thank you!

[sadasant]

I was able to reproduce this by doing the following:

• Made a new folder with a simple package.json
  
  
  
  • Setting @azure/identity to a specific version...
  
  
  • First to 1.2.0-beta.1.
  
  
  • Then to 1.2.0-alpha.20200924.1.
  
  
  • Then to something along the lines of ./azure-sdk-for-js/sdk/identity/identity/azure-identity-1.2.0-beta.2.tgz.
  
  
• Used a simple script that creates a DefaultAzureCredential() and calls to getToken.
  
  
  
  • I got the same result calling getToken with either of:
  
  
  • https://management.core.windows.net/.default and
  
  
  • https://management.core.windows.net.
  
  
• I made this this folder with the same contents both in PowerShell and the WSL.
  
  
  
  • I wasn't able to reproduce this in PowerShell.
  
  
  • I was able to reproduce this consistently in the WSL (Debian & bash).
  
  

For concrete details, the package.json in the WSL environment currently looks like this:

{
  "name": "identity-test",
  "version": "1.0.0",
  "description": "",
  "main": "index.js",
  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1"
  },
  "author": "",
  "license": "ISC",
  "dependencies": {
    "@azure/identity": "./azure-sdk-for-js/sdk/identity/identity/azure-identity-1.2.0-beta.2.tgz",
    "@azure/keyvault-keys": "^4.1.0"
  }
}

The script in question is in JavaScript, and the contents were:

const identity = require("@azure/identity");

async function main() {
  const cred = new identity.DefaultAzureCredential();
  const token = await cred.getToken("https://management.core.windows.net/.default");
  console.log({ token });
}

main().then(console.log).catch((e) => console.error(e));

IMPORTANT:

The error code I was able to see in the WSL was ECONNREFUSED. @mhoeger is describing an error that shows EHOSTDOWN. I'm adding both as part of my fix (in this PR).

About ECONNREFUSED and EHOSTDOWN:

I believe both convey the same kind of errors we're trying to identify. To back me up, I have the following:

Oracle docs (link), for example, say:

• ECONNREFUSED: The host refused service. This happens when a server process is not present at the requested address.
• EHOSTDOWN: These errors are caused by status information delivered by the underlying communication interface.

NodeJS's source shows:

• EHOSTDOWN seems to be an implementation detail (link) in some OS systems / environments.
• Internals (link) show more information regarding these error codes, based on which I believe these two should be considered about the same for the purposes of skipping these error cases while the credential runs underneath the DefaultAzureCredential:
  
  
  
  • XX(ECONNREFUSED, "connection refused")
  
  
  • XX(EHOSTDOWN, "host is down")
  
  

My remaining question is:
What other error codes should we consider the same?, or perhaps is there a better way to capture this than to check for the error codes?

I'll be in contact with my team today to see if they have a better response than me.

[mhoeger]

You rock! Thanks @sadasant !

[sadasant]

@mhoeger Hello, Marie!

We have released @azure/[email protected]! Please install it and try again. Let us know if that version provides the expected relief!

Have a good week 👋