Salt: ssh_known_hosts.present appends public key when that key already exists in the known_hosts file

[jaloren]

salt version: 2014.1.5
OS: CentOS 6.4

Using the ssh_known_hosts.present in a state file.

http://salt.readthedocs.org/en/v2014.1.1/ref/states/all/salt.states.ssh_known_hosts.html?highlight=known_hosts#salt.states.ssh_known_hosts.present

Here is the state file.

https://gist.github.com/jaloren/d074e69f7c64ea798031

Expectation is that the function behave like file.append:

1. check the known_hosts file for the public key.
2. If public key is found, then do nothing.
3. if the public key is not found, then append the public key to the file.

However, instead the function appends the public key no matter what. This means that the public key is appended to the known_hosts file after each state run, regardless of whether the entry is already there. As a consequence, the known_hosts file is filled with multiple entries, where each entry is of the same public key.

[basepi]

Thanks for the report. I think your expected behavior is the expected behavior, but I'm guessing we're not parsing it quite right and so are not detecting the fact that the key already exists.

[steverweber]

might be related to https://github.com/saltstack/salt/issues/19917

[Reiner030]

While searching for similar open bugs I found this... But this one is fixed and can be closed ? ;)

[basepi]

@Reiner030 have you tested and this is fixed?

[Reiner030]

in my cases I have only 1 ssh hostkey line per hostnames on every run.

Thats for my configuration setup also a problem because I can't save rsa,dsa, ecdsa (and in Debian 8 ed25519) pubkeys with same hostname in this file.
So I had to program if-then-else cases around it in the default ssh order ecdsa, ed25519, rsa, dsa to save the first key I found created.

[hackel]

Just encountered this issue, running 2015.5.2. Minion user's ~/.ssh/known_hosts quickly fills up with identical entries if hash_hostname: false. If I also set a key, things get really wonkey, with both hashed and unhashed hostnames being added, sometimes with multiple hashes. Luckily I don't care if it's hashed, I was just setting that for testing purposes.

[marco-websource]

Same problem here, running 2015.5.3 on Debian 8.1. At first it seemed to work fine but suddenly ssh_known_hosts.present keeps adding identical entries. I also noticed that since this problem started all added entries have unhashed hostnames, even if hash_hostname: true! Duplicates are added regardless if hash_hostname is true or false.

UPDATE: I think I found the cause of my problem: I had set enc: ecdsa according to the documentation. However, my version of OpenSSH (OpenSSH_6.7p1 Debian-5), seems to require the full key spec in the ssh_known_hosts file. When setting enc: ecdsa-sha2-nistp256 the problem is gone. I think to fix this enc should be sanitized with _refine_enc(enc) in set_known_host() at https://github.com/saltstack/salt/blob/develop/salt/modules/ssh.py#L941 . Please note that this is only true if a key is specified. If no key is specified, recv_known_host() is used to retrieve the key, which I believe requires enc = ecdsa and then returns the full key specification.

While investigating this problem, I found another issue when a key instead of a fingerprint is specified: Newly added keys are not stored with hashed hostnames. The hostname of a key is only converted to a hashed hostname after another key has been added later. I think this is because ssh.hash_known_hosts() in set_known_host() at https://github.com/saltstack/salt/blob/develop/salt/modules/ssh.py#L971 is called before the new line with the unhashed hostname is added. I think this can be fixed by moving ssh.hash_known_hosts() to the end of the of the function. Again, I believe this is only true when a key is specified since recv_known_host() should return the hashed key directly.

Shall I file new bug reports for these issues?

[fenderle]

Same problem here (2015.8.3) - everything ok with ecdsa. When I use enc: ecdsa-sha2-nistp256 the problem starts. And my OpenSSH also wants the full ecdsa name.

[hugochinchilla]

this is causing me so much pain, will it be fixed anytime soon?

[cmeisinger]

Confirmed and still present.

My 300k known_hosts file is anxious for this fix. :)

[babilen5]

Just confirmed on IRC to be present in 2016.3.1. This makes it incredibly hard for people to securely populate known_hosts.

[AstraLuma]

I just stumbled on this with 2016.11.2. Looking at the source, it seems to stem from the fact that get_known_host() (of salt/modules/ssh.py) will only return one host line. I'm trying to add several keys for the same host (because it has rsa, ecdsa, and ed25519 keys).

[rallytime]

This should be fixed with #43476.