Azure-cli: az acs kubernetes get-credentials fails authentication

Created on 11 Dec 2017 · 9Comments · Source: Azure/azure-cli

On a fresh install of azure-cli via homebrew, it installs python3 as a dependency, and successfully installs. I am able to login using az login, and can then subsequently run az acs list -o table and see my clusters (full output below). Then when trying to get the credentials, authentication fails for some unknown reason - az acs kubernetes get-credentials.

I was able to run the same process on a Linux system using yum to install az and it worked fine. It did use Python 2.7 rather than 3.6.

Environment summary

OSX 10.13.1
installed via homebrew
version 2.0.22
zsh using iTerm2

az --version
azure-cli (2.0.22)

acr (2.0.16)
acs (2.0.21)
advisor (0.1.0)
appservice (0.1.21)
backup (1.0.3)
batch (3.1.7)
batchai (0.1.3)
billing (0.1.6)
cdn (0.0.10)
cloud (2.0.10)
cognitiveservices (0.1.9)
command-modules-nspkg (2.0.1)
configure (2.0.12)
consumption (0.2.0)
container (0.1.14)
core (2.0.22)
cosmosdb (0.1.15)
dla (0.0.15)
dls (0.0.18)
eventgrid (0.1.5)
extension (0.0.6)
feedback (2.0.6)
find (0.2.7)
interactive (0.3.11)
iot (0.1.14)
keyvault (2.0.14)
lab (0.0.13)
monitor (0.0.13)
network (2.0.18)
nspkg (3.0.1)
profile (2.0.15)
rdbms (0.0.9)
redis (0.2.10)
reservations (0.1.0)
resource (2.0.19)
role (2.0.15)
servicefabric (0.0.6)
sql (2.0.16)
storage (2.0.20)
vm (2.0.19)

Python location '/usr/local/opt/python3/bin/python3.6'
Extensions directory '/Users/derek/.azure/cliextensions'

Python (Darwin) 3.6.3 (default, Oct  4 2017, 06:09:38)
[GCC 4.2.1 Compatible Apple LLVM 9.0.0 (clang-900.0.37)]

CLI Output

$$ az acs list -o table
Location        Name          ProvisioningState    ResourceGroup
--------------  ------------  -------------------  -------------------
northcentralus  abcd  Succeeded            1234

$$ az acs kubernetes get-credentials -n abcd -g 1234
Authentication failed.
Traceback (most recent call last):
  File "/usr/local/Cellar/azure-cli/2.0.22/libexec/lib/python3.6/site-packages/azure/cli/main.py", line 36, in main
    cmd_result = APPLICATION.execute(args)
  File "/usr/local/Cellar/azure-cli/2.0.22/libexec/lib/python3.6/site-packages/azure/cli/core/application.py", line 216, in execute
    result = expanded_arg.func(params)
  File "/usr/local/Cellar/azure-cli/2.0.22/libexec/lib/python3.6/site-packages/azure/cli/core/commands/__init__.py", line 381, in __call__
    return self.handler(*args, **kwargs)
  File "/usr/local/Cellar/azure-cli/2.0.22/libexec/lib/python3.6/site-packages/azure/cli/core/commands/__init__.py", line 630, in _execute_command
    reraise(*sys.exc_info())
  File "/usr/local/Cellar/azure-cli/2.0.22/libexec/lib/python3.6/site-packages/six.py", line 693, in reraise
    raise value
  File "/usr/local/Cellar/azure-cli/2.0.22/libexec/lib/python3.6/site-packages/azure/cli/core/commands/__init__.py", line 612, in _execute_command
    result = op(client, **kwargs) if client else op(**kwargs)
  File "/usr/local/Cellar/azure-cli/2.0.22/libexec/lib/python3.6/site-packages/azure/cli/command_modules/acs/custom.py", line 866, in k8s_get_credentials
    _k8s_get_credentials_internal(name, acs_info, path, ssh_key_file)
  File "/usr/local/Cellar/azure-cli/2.0.22/libexec/lib/python3.6/site-packages/azure/cli/command_modules/acs/custom.py", line 887, in _k8s_get_credentials_internal
    '.kube/config', path_candidate, key_filename=ssh_key_file)
  File "/usr/local/Cellar/azure-cli/2.0.22/libexec/lib/python3.6/site-packages/azure/cli/command_modules/acs/acs_client.py", line 72, in secure_copy
    ssh.connect(host, username=user, pkey=pkey, sock=proxy)
  File "/usr/local/Cellar/azure-cli/2.0.22/libexec/lib/python3.6/site-packages/paramiko/client.py", line 424, in connect
    passphrase,
  File "/usr/local/Cellar/azure-cli/2.0.22/libexec/lib/python3.6/site-packages/paramiko/client.py", line 714, in _auth
    raise saved_exception
  File "/usr/local/Cellar/azure-cli/2.0.22/libexec/lib/python3.6/site-packages/paramiko/client.py", line 691, in _auth
    self._transport.auth_publickey(username, key))
  File "/usr/local/Cellar/azure-cli/2.0.22/libexec/lib/python3.6/site-packages/paramiko/transport.py", line 1450, in auth_publickey
    return self.auth_handler.wait_for_response(my_event)
  File "/usr/local/Cellar/azure-cli/2.0.22/libexec/lib/python3.6/site-packages/paramiko/auth_handler.py", line 226, in wait_for_response
    raise e
paramiko.ssh_exception.AuthenticationException: Authentication failed.

ACS Service Attention

Source

derekperkins

👍1

Most helpful comment

I have the same issue as @DaveVoyles. Note that these exact commands are mentioned in the MS walkthrough document https://docs.microsoft.com/en-us/azure/container-service/kubernetes/container-service-kubernetes-windows-walkthrough, which does not work because of this problem.

ursqontis on 21 Aug 2018

👍2

All 9 comments

I tried uninstalled the homebrew version of azure-cli and python3 and ran the curl install. Now running on the default python 2.7 and rerunning the same login and acs list process, I get apparently the same error (slightly different stack trace).

Authentication failed.
Traceback (most recent call last):
  File "/Users/derek/azure-cli/lib/python2.7/site-packages/azure/cli/main.py", line 36, in main
    cmd_result = APPLICATION.execute(args)
  File "/Users/derek/azure-cli/lib/python2.7/site-packages/azure/cli/core/application.py", line 216, in execute
    result = expanded_arg.func(params)
  File "/Users/derek/azure-cli/lib/python2.7/site-packages/azure/cli/core/commands/__init__.py", line 381, in __call__
    return self.handler(*args, **kwargs)
  File "/Users/derek/azure-cli/lib/python2.7/site-packages/azure/cli/core/commands/__init__.py", line 630, in _execute_command
    reraise(*sys.exc_info())
  File "/Users/derek/azure-cli/lib/python2.7/site-packages/azure/cli/core/commands/__init__.py", line 612, in _execute_command
    result = op(client, **kwargs) if client else op(**kwargs)
  File "/Users/derek/azure-cli/lib/python2.7/site-packages/azure/cli/command_modules/acs/custom.py", line 866, in k8s_get_credentials
    _k8s_get_credentials_internal(name, acs_info, path, ssh_key_file)
  File "/Users/derek/azure-cli/lib/python2.7/site-packages/azure/cli/command_modules/acs/custom.py", line 887, in _k8s_get_credentials_internal
    '.kube/config', path_candidate, key_filename=ssh_key_file)
  File "/Users/derek/azure-cli/lib/python2.7/site-packages/azure/cli/command_modules/acs/acs_client.py", line 72, in secure_copy
    ssh.connect(host, username=user, pkey=pkey, sock=proxy)
  File "/Users/derek/azure-cli/lib/python2.7/site-packages/paramiko/client.py", line 424, in connect
    passphrase,
  File "/Users/derek/azure-cli/lib/python2.7/site-packages/paramiko/client.py", line 714, in _auth
    raise saved_exception
AuthenticationException: Authentication failed.

Environment

az --version
azure-cli (2.0.22)

acr (2.0.16)
acs (2.0.21)
advisor (0.1.0)
appservice (0.1.21)
backup (1.0.3)
batch (3.1.7)
batchai (0.1.3)
billing (0.1.6)
cdn (0.0.10)
cloud (2.0.10)
cognitiveservices (0.1.9)
command-modules-nspkg (2.0.1)
configure (2.0.12)
consumption (0.2.0)
container (0.1.14)
core (2.0.22)
cosmosdb (0.1.15)
dla (0.0.15)
dls (0.0.18)
eventgrid (0.1.5)
extension (0.0.6)
feedback (2.0.6)
find (0.2.7)
interactive (0.3.11)
iot (0.1.14)
keyvault (2.0.14)
lab (0.0.13)
monitor (0.0.13)
network (2.0.18)
nspkg (3.0.1)
profile (2.0.15)
rdbms (0.0.9)
redis (0.2.10)
reservations (0.1.0)
resource (2.0.19)
role (2.0.15)
servicefabric (0.0.6)
sql (2.0.16)
storage (2.0.20)
vm (2.0.19)

Python location '/Users/derek/azure-cli/bin/python'
Extensions directory '/Users/derek/.azure/cliextensions'

Python (Darwin) 2.7.10 (default, Jul 15 2017, 17:16:57)
[GCC 4.2.1 Compatible Apple LLVM 9.0.0 (clang-900.0.31)]

derekperkins on 11 Dec 2017

More information... I tried creating a new cluster with az aks, which succeeded, and the subsequent az aks get-credentials --resource-group 1234 --name abcd did work, so I'm only seeing the error with az acs.

derekperkins on 11 Dec 2017

@derekperkins have you tried using --ssh-key-file argument pointing to correct ssh private key in your az acs get-credentials ...? I was having the same error as you, but I did merge the context in my kubeconfig using that argument

ghost on 18 Dec 2017

@Andre-Freitas Does acs use different credential validation than aks?

derekperkins on 18 Dec 2017

👍1

@derekperkins when i created my acs i used az acs create --orchestrator-type kubernetes --resource-group <resource group name> --name <cluster name> --ssh-key-value <public ssh key> but you can also use this az acs create --orchestrator-type kubernetes --resource-group <resource group name> --name <cluster name> --generate-ssh-keys

ghost on 19 Dec 2017

For v1 container service created through az acs create, like @Andre-Freitas mentioned, to get the credential the original ssh key file is needed to open a SSH connection to the host so to retrieve the creds configs. For v2 container service created through az aks create, the ssh file is no longer needed to get the credentials as all are available through the new Azure Kubenetes Management APIs

yugangw-msft on 4 Jan 2018

I'm kind of confused by this too.

I ran this command:

az group create --name dv-acs-rg --location eastus```


``` az acs create --agent-vm-size Standard_NC6 --resource-group dv-acs-rg --name dv-acs 
--orchestrator-type Kubernetes --agent-count 1 
--location esatus --generate-ssh-keys

And afterwards:

az acs kubernetes get-credentials --name dv-acs --resource-group dv-acs-rg

When it asked:

Password for private key:

What password? For which private key? I thought it generated them for me?