Awx: Enable signed SSH certificates as Machine credentials

[astraios]

ISSUE TYPE

• Feature Idea

COMPONENT NAME

• awx_tasks

SUMMARY

This idea is to use signed SSH certificates, provided by Hashicorp Vault.
You can check https://www.vaultproject.io/docs/secrets/ssh/signed-ssh-certificates.html.

I want to stop deploying public keys, and use short-lived signed certificates provided.

[wenottingham]

The way this would have to work for in-Tower machine creds is an optional field to hold the signed cert for the key. It would then need templated out when adding to the ssh agent.

[RRAlex]

Indeed, even if I encode the *-cert.pub in PEM format, AWX will say it doesn't support public key, but since the private key is unmodified when signed, there is no way to deploy a single CA in the authorized_keys of your machines to than have multiple keys used by different users within AWX...

[ryanpetrello]

@astraios @RRAlex,

We're currently working on implementing this (along with a more general plugin framework for credential lookups) here: https://github.com/ansible/awx/pull/3098

Would you mind reading over this, and leaving some feedback about whether you think it would work for you?
https://github.com/ansible/awx/pull/3098/files#diff-7d017aa98d3cddf99ff85fff0da8d743

...specifically, the section labeled HashiCorp Vault SSH Secrets Engine.

This comment about the general use cases we intend to support may also interest you (I'd love your feedback):
https://github.com/ansible/awx/pull/3098/files#r260528206

[RRAlex]

In my case, the problem wasn't about integrating another secret engine, but simply being able to insert CA signed keys in the key field to be encrypted and kept as secret within AWX's database.
Vault is nice, but it is a whole other beast that I wouldn't implement simply for this specific need though.
Cheers!

[jakemcdermott]

@RRAlex You'll be able to provide a private key and signed cert directly without using an external secrets engine. There will be an additional field on the machine credential form for a signed ssh certificate.

[jakemcdermott]

resolved by https://github.com/ansible/awx/pull/3098

[bkmeneguello]

When this new feature will be available?

[wenottingham]

... as noted, the PR was merged into AWX in April and is available now. If you are asking about something that pulls from AWX such as Ansible Tower, check with your Red Hat rep.

[bkmeneguello]

I'm asking when this will be available not in "devel" but as a stable tag. Sorry if I didn't make my self clear.
I'm using v4.0.0. I think this will be available in v4.1.0. There is a release date?

[wenottingham]

I'm not directly on the eng side, but I believe in the next couple of weeks.

[bkmeneguello]

I've tried this on the new 5.0.0 release but I cannot make it work.
I have a valid signed certificate (I can connect using it with ssh) but whenever I try to use in AWX it give me "connection denied". I tested the connectivity adding the public key to "authorized_keys" and, without any credential modification, it worked.
It seems like the AWX ssh client cannot lookup the signed certificate properly. I'm trying to debug this right but I have low familiarity with AWX code base.
[edit]
I think the private key is being written to "//env/ssh_key" while the certificate is written to "//credential_-cert.pub". I don't fully understand how ssh resolves the public key, but seems it uses the private key name plus "-cert" in the same path.
[edit2]
I checked the "ssh-add" docs and the previous assumption is correct. From docs:

After loading a private key, ssh-add will try to load corresponding certificate information from the filename obtained by appending -cert.pub to the name of the private key file. Alternative file names can be given on the command line.

Since ansible-runner is not giving alternative certificate names on command line, it must be placed into "artifacts/" with name "ssh_key_data-cert.pub"
[edit3]
I did the workaround and added the following lines to tasks.py#829 and it worked.

for credential, data in private_data.get('certificates', {}).items():
    name = 'ssh_key_data-cert.pub'
    path = os.path.join(private_data_dir, 'artifacts', str(instance.id))
    if not os.path.exists(path):
        os.makedirs(path, mode=0o700)
    path = os.path.join(path, name)

I know this is not the right way to fix, is just to check the solution.
[edit4]
Issue #4139 created