Paramiko: Cannot parse OpenSSH 6.5 formatted private keys

[kabel]

The key format described at https://github.com/openssh/openssh-portable/blob/master/PROTOCOL.key is not currently supported causing key authentication problems when that format is used. The current parser erroneously assumes that the key file is unencrypted.

Related to #387

[mchlt]

Looking into this issue.
Some additional info about the key format here:
http://www.tedunangst.com/flak/post/new-openssh-key-format-and-bcrypt-pbkdf

[mchlt]

OK, i now have some working code that correctly reads the base64 blobs from a new OPENSSH type key.
Unfortunately, OpenSSH have chosen a rather exotic key derivation function to derive the key from the passphrase: bcrypt_pbkdf
The only Python implementation of that that I could find is in py_bcrypt (https://github.com/grnet/python-bcrypt). I'll use that for now to see if I can actually do the decryption.

Wondering what's paramiko's stance on introducing new module dependencies? I'm thinking maybe do the import in a try..except and then throwing an exception only if someone tries to load an encrypted new format key?

[bitprophet]

@mchlt We do the "try/except/complain" for other optional deps right now, such as python-gssapi for GSSAPI support (docs). So we don't want to do it willy-nilly, but there's precedent.

That said, "a new privkey format used by OpenSSH" feels more mainstream a feature (at least going forwards) than Kerberos/GSSAPI support, so I lean towards "if we can get this implemented before I finish the other work for 2.0, let's put python-bcrypt in that release as a hard dependency".

If you haven't, you might also look to see if https://github.com/pyca/cryptography has it (this mentions pbkdf2 but I'm not crypto-fluent enough to tell if it's truly the same as python-bcrypt's C-ext) or is willing to add it, since paramiko 2.0 will be relying on cryptography anyways.

[bitprophet]

Also, thanks a lot for looking into this! :heart:

[bitprophet]

Continuing discussion in here since not 100% sure what'll happen to #618 (it might want closing & replacing with another PR):

• #618 notes that Cryptography doesn't support bcrypt (specifically bcrypt_pbkdf). I'm curious whether @alex / @reaperhulk think it's worth adding to Cryptography at any point? Don't see anything over there implying such, but always nice to ask.
• Assuming that's not in the works, I'm still reasonably okay with adding the python-bcrypt dependency instead. (TBH, even if Cryptography added it I wouldn't be surprised if they wrapped python-bcrypt itself, so...)
• In either case, however, #618 doesn't merge cleanly with #394 which is about to land in master.
  
  
  
  • If @mchlt has time in the very near future to address that (ideally a few days; yes, asking another to get back to me in days when I tend to get back in months, is horrid 😖 ), I can squeeze this into 2.0 as originally planned, and add python-bcrypt as another new install-time dependency.
  
  
  • If he doesn't (which would be fine!) I'll put it into the 3.0 bucket, which is basically what 2.0 used to be.
  
  
  • That will still likely come out sooner instead of later (again see #394 for thoughts on release junk), FYI - not planning it to be another very long term major release but a short/mid term one.
  
  

[reaperhulk]

We'd like to support bcrypt directly, but right now cryptography is (mostly) tied to OpenSSL which doesn't support eksblowfish. We do have pyca/bcrypt, which will shortly be getting wheels for OS X and even linux (via manylinux1, see PEP 513).

[bitprophet]

@reaperhulk Huh, neat - I'd still feel better about depending on a PyCA-blessed/implemented bycrypt module vs one that seems to largely be a one-off (not to diss python-bcrypt / py-bcrypt of course, but.) Thanks!

/cc @mchlt - if pyca/bcrypt serves your needs as well as the other one, maybe switching to it would be something to do while updating.

[mchlt]

Whoa, all going fast suddenly :)
Will try to see what I can do tonight

Unfortunately pyca/bcrypt does not have the bcrypt pbkdf function which is
needed for the new openssh private key file format.

Op ma 25 apr. 2016 02:19 schreef Jeff Forcier [email protected]:

@reaperhulk https://github.com/reaperhulk Huh, neat - I'd still feel
better about depending on a PyCA-blessed/implemented bycrypt module vs one
that seems to largely be a one-off (not to diss python-bcrypt / py-bcrypt
of course, but.) Thanks! /cc @mchlt https://github.com/mchlt - if
pyca/bcrypt serves your needs as well as the other one, maybe switching to
it would be something to do while updating.

—
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub
https://github.com/paramiko/paramiko/issues/602#issuecomment-214073471

[reaperhulk]

@mchlt Interesting, we definitely don't. I'll take a look at what it would require to do that today. Unfortunately the underlying C lib we use (crypt_blowfish) hasn't been updated to add support for that.

[reaperhulk]

As an update, PyCA bcrypt 3.1.0 is out which provides a bcrypt_pbkdf and ships wheels for Windows, Mac, and manylinux1 so in many cases you won't need a compiler at all!

[ploxiln]

Support for the new bcrypt-protected key format was added, just for ed25519 keys, in #972

(support is still missing for rsa keys in the new bcrypt-based format - I actually use such a key, but via an ssh-agent)

[trou]

Just chiming in for the new RSA key format, it would be nice to have it, as it is way more secure that the old format.

[raoulh]

Paramiko needs this new key format as Openssh defaults to this format since the latest release.

https://www.openssh.com/releasenotes.html

• ssh-keygen(1): write OpenSSH format private keys by default
  instead of using OpenSSL's PEM format. The OpenSSH format,
  supported in OpenSSH releases since 2014 and described in the
  PROTOCOL.key file in the source distribution, offers substantially
  better protection against offline password guessing and supports
  key comments in private keys.

[TurboTurtle]

Just checking in here, is the issue with #618 still the same as noted in the April 24 2016 comment above?

Is there any timeframe on paramiko supporting the new key format?

[jaredhobbs]

I have a PR open #1343 based on the code in #618 that will merge cleanly with master.

[OddBloke]

The OpenSSH key format became the default generated by ssh-keygen in version 7.8, so people are going to start hitting this issue more and more in the wild (without having explicitly opted in to the new format).

In my specific case, Ubuntu 19.04 (Disco Dingo), the currently in-development release of Ubuntu, includes a post-7.8 version (7.9p1 at the time of writing). Ubuntu users who upgrade in April are going to start hitting this issue.

[hunterirving]

I would also like to see support for this new format.

[bitprophet]

If anyone else has the spare cycles to check up on the years-old assertions I made early in this ticket, re: whether Cryptography has what we need now, that'd be awesome. I'm optimistic that this is possible to do now, if it hasn't already been done in a fork somewhere (looking at @ploxiln whose fork I do not have time to go over in detail yet).

[bitprophet]

Also cc @alex and @reaperhulk as usual on that front ❤️

[bitprophet]

Also VERY briefly skimming my old comments here, I'd put this in 3.0's milestone on assumption that this requires a new dependency (python-bcrypt). In the intervening years I've both relaxed my stance on that topic, and again, Cryptography has gone through a lot of development.

I'm going to move this into the p1 milestone with those things in mind - I expect we could do this in Paramiko 2.6 or whatever without making anybody too unhappy, even if it does mean a new dependency is added. Folks grabbing new Paramiko minor releases _and_ not doing it for the new key type _and_ being really mad about some new-to-them dependency, seems like it'd be a very small demographic.

[ploxiln]

the bcrypt dependency was already added in paramiko 2.2.0 for Ed25519 keys - https://github.com/paramiko/paramiko/pull/972