Paramiko: Paramiko regards valid RSA private keys as invalid

Created on 29 May 2014 · 17Comments · Source: paramiko/paramiko

OS: RHEL 6.4, product version: python-paramiko-1.7.5-2.1.el6.noarch

I did not manage to pass to paramiko RSA private key issued by oVirt manager CA (also known as Red Hat Enterprise Virtualization Manager).

[root@vitaly registration]# ssh rhevaio
root@rhevaio's password: 
Last login: Thu May 29 15:03:33 2014 from 10.0.55.84
[root@rhevaio ~]# scp /etc/pki/ovirt-engine/keys/engine_id_rsa [email protected]:/tmp/ololo/engine_id_rsa_RHEVAIO
[email protected]'s password: 
engine_id_rsa                                                                                                                                                                100% 1708     1.7KB/s   00:00

In [1]: import paramiko

In [2]: pk = paramiko.RSAKey.from_private_key_file("/tmp/ololo/engine_id_rsa_RHEVAIO")
---------------------------------------------------------------------------
SSHException                              Traceback (most recent call last)

/home/vitaly/progs/registration/<ipython console> in <module>()

/usr/lib/python2.6/site-packages/paramiko/pkey.pyc in from_private_key_file(cls, filename, password)
    195         @raise SSHException: if the key file is invalid
    196         """
--> 197         key = cls(filename=filename, password=password)
    198         return key
    199     from_private_key_file = classmethod(from_private_key_file)

/usr/lib/python2.6/site-packages/paramiko/rsakey.pyc in __init__(self, msg, data, filename, password, vals, file_obj)
     49             return
     50         if filename is not None:
---> 51             self._from_private_key_file(filename, password)
     52             return
     53         if (msg is None) and (data is not None):

/usr/lib/python2.6/site-packages/paramiko/rsakey.pyc in _from_private_key_file(self, filename, password)
    162 
    163     def _from_private_key_file(self, filename, password):
--> 164         data = self._read_private_key_file('RSA', filename, password)
    165         self._decode_key(data)
    166 

/usr/lib/python2.6/site-packages/paramiko/pkey.pyc in _read_private_key_file(self, tag, filename, password)
    277         """
    278         f = open(filename, 'r')
--> 279         data = self._read_private_key(tag, f, password)
    280         f.close()
    281         return data

/usr/lib/python2.6/site-packages/paramiko/pkey.pyc in _read_private_key(self, tag, f, password)
    287             start += 1
    288         if start >= len(lines):
--> 289             raise SSHException('not a valid ' + tag + ' private key file')
    290         # parse any headers first

    291         headers = {}

SSHException: not a valid RSA private key file

However, I can use this key to start trivial ssh session with oVirt server. It's absolutely OK:

ssh -i /tmp/ololo/engine_id_rsa_RHEVAIO root@<host>

See below the body of the key (I've changed some random bytes in signature due to security reasons). Have no idea why doesn't it match Paramiko's standards :

Source

vitalyisaev2

👍3

Most helpful comment

This is a problem with recent macOS, whose OpenSSL ssh-keygen has changed the default format. Now a generated SSH key starts with:

-----BEGIN OPENSSH PRIVATE KEY-----

instead of the supported

-----BEGIN RSA PRIVATE KEY-----

To generate a supported key, use the following command:

ssh-keygen -t rsa -b 4096 -C "[email protected]" -m PEM

hyperknot on 15 May 2019

👍39 🎉9 😕3 ❤1

All 17 comments

Normally, keys start with this line, and this is what paramiko is checking for:

-----BEGIN RSA PRIVATE KEY-----

But as SSH does not seem to care if "RSA " is there or not, paramiko shouldn't, either, I guess.

rfjakob on 6 Jun 2014

If you're truly on Paramiko 1.7.5, that version is quite ancient at this point and wholly unsupported :( sorry! If you can recreate this on an edition newer than 1.8 or so, please feel free to reopen. thanks!

bitprophet on 9 Aug 2014

I am experiencing this problem with paramiko in Debian Jessie (python-paramiko 1.15.1-1) with a key generated with: ssh-keygen -b4096 and adding a passphrase. _The problem only occurs when applying a passphrase to the key._

I am using paramiko through duplicity and get the following error message
BackendException: ssh connection to test@remote failed: not a valid EC private key file
I then found out through the fine manual that duplicity _won't allow me to use passphrased SSH key files_ and still had this thread open, so I figured I'll report my findings here.

I have not read enough about paramiko to know whether you intend to support passphrased SSH key files or not, but at least this post may offer an explanation to the original poster as well as to any future web searches that experience the "not a valid RSA private key file" or whatever format was expected.

ghost on 8 Nov 2014

How about SSH2 PUBLIC KEY connection?

Ghost0817 on 13 Apr 2015

Still happen with 2.0.2.

Env

version

paramiko==2.0.2

Linux Mint 18

Key generation

ssh-keygen -q \
            -t rsa \
            -b 4096 \
            -f "${SSH_KEYFILE}" \
            -N "${EMPTY}" \
            -O permit-port-forwarding \
            -C "Automatically generated by MAST script"

Connection code

import os
import paramiko

def open_ssh_connection(hostname, port=22, ssh_key=None):
    paramiko.util.log_to_file('/tmp/ssh.log')  # sets up logging

    try:
        client = paramiko.SSHClient()
        if ssh_key:
            print('key: ' + ssh_key )
            client.load_system_host_keys(ssh_key)
            client.load_host_keys(ssh_key)
        client.set_missing_host_key_policy(paramiko.AutoAddPolicy())
        client.connect(hostname, port=port, timeout=0.3, username='toto')

    except paramiko.BadHostKeyException as e:
        raise Exception('BadHostKeyException on ' + hostname)
    except paramiko.AuthenticationException as e:
        raise Exception('AuthenticationException on ' + hostname)
    except paramiko.SSHException as e:
        raise Exception('SSHException on ' + hostname)

    return client

my_key = os.path.expanduser('/home/mast/.ssh/id_rsa.toto')
hostname = '192.168.2.231'
connection = open_ssh_connection(hostname)
stdin, stdout, stderr = connection.exec_command('ls -l')

for line in stdout:
    print(line)

Error

key: /home/mast/.ssh/id_rsa.toto
Traceback (most recent call last):
File "/tmp/ssh.py", line 15, in open_ssh_connection
client.connect(hostname, port=port, timeout=0.3, username='toto')
File "/usr/local/lib/python3.5/site-packages/paramiko/client.py", line 380, in connect
look_for_keys, gss_auth, gss_kex, gss_deleg_creds, gss_host)
File "/usr/local/lib/python3.5/site-packages/paramiko/client.py", line 603, in _auth
raise saved_exception
File "/usr/local/lib/python3.5/site-packages/paramiko/client.py", line 577, in _auth
key = pkey_class.from_private_key_file(filename, password)
File "/usr/local/lib/python3.5/site-packages/paramiko/pkey.py", line 196, in from_private_key_file
key = cls(filename=filename, password=password)
File "/usr/local/lib/python3.5/site-packages/paramiko/rsakey.py", line 45, in __init__
self._from_private_key_file(filename, password)
File "/usr/local/lib/python3.5/site-packages/paramiko/rsakey.py", line 163, in _from_private_key_file
data = self._read_private_key_file('RSA', filename, password)
File "/usr/local/lib/python3.5/site-packages/paramiko/pkey.py", line 268, in _read_private_key_file
data = self._read_private_key(tag, f, password)
File "/usr/local/lib/python3.5/site-packages/paramiko/pkey.py", line 277, in _read_private_key
raise SSHException('not a valid ' + tag + ' private key file')
paramiko.ssh_exception.SSHException: not a valid RSA private key file

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/tmp/ssh.py", line 30, in <module>
connection = open_ssh_connection(hostname, port)
File "/tmp/ssh.py", line 22, in open_ssh_connection
raise Exception('SSHException on ' + hostname)
Exception: SSHException on 192.168.2.231

edouard-lopez on 29 Sep 2016

@edouard-lopez - Seems to be some confusion between server identification keys (host keys) and user identification keys in the code you posted. Host keys are typically stored in ~/.ssh/known_hosts file and have a list of public keys that have been accepted during the first ever connection to the host by the client (presumably after verifying the fingerprint by some other means). There is also a system-wide file, /etc/ssh/ssh_known_hosts that can provide a list of trusted host keys available for all users. These are the two files that would typically be passed to client.load_host_keys() and client.load_system_host_keys(), not a private key file.

When you connect to a server via SSH, the client uses the remote server's host key to verify that the server identity matches the host that was trusted from an earlier connection with the matching (and accepted) public key on file in known_hosts. If there is no matching host entry in the known_hosts file, the default behavior is to _not_ trust the unknown host, and drop the connection. The AutoAddPolicy is used to change that default behavior to that the key will be accepted when there is no matching entry for the current host.

For user authentication, the client can use a private key instead of a password to login to the remote host, as long as the ssh server allows it, and the public key is in the ~/.ssh/authorized_keys file on the remote host. This is the stage where you would use the id_rsa.toto file as the key_filename parameter to the client.connect() call.

I'd also recommend that you test the key based authentication with ssh -i ~/.ssh/id_rsa.toto toto@host to confirm that the public key is in the right place and the file/directory permissions are correct. If the configuration setup won't work for the normal ssh client, it's not going to work under paramiko

radssh on 29 Sep 2016

👍2

Get it working with:

def fetch_netmask(hostname, port=22):
    private_key = os.path.expanduser('~/.ssh/id_rsa.custom')
    connection = open_ssh_connection('coaxis', hostname, port=port, key=private_key)

    get_netmask = ("ip -oneline -family inet address show | grep {}").format(hostname)
    stdin, stdout, stderr = connection.exec_command(get_netmask)
    address = parse_address(hostname, stdout)
    connection.close()

    return address


def open_ssh_connection(username, hostname, port=22, key=None):
    client = paramiko.SSHClient()
    client.set_missing_host_key_policy(paramiko.AutoAddPolicy())
    client.connect(hostname, port=port, timeout=0.3, username=username, key_filename=key)

    return client

edouard-lopez on 29 Sep 2016

👍2

I am facing the same problem with paramiko 2.4.0. I am with django + apache server + CentOS 7.

I think it is something with the format, not of permission, not for key itself, because with the same key, I can:

connect to the server to execute command with ssh in bash. (ssh -i /path/to/private-key host)
connect to the server to execute command with python manage.py runserver 0.0.0.0:8080, without changing the code

But when I switch to httpd, the error occurs. If, for anything, it is a permission problem, like it was before I modified the permission and owner of the file and all its parent dir, it was another exception Permission denied.. Now it is not the case.

The exception I got was:

SSHException('not a valid OPENSSH private key file',)

The private key is like:

-----BEGIN RSA PRIVATE KEY-----
MIIEowIBAAKCAQEA0SLRpBwbygz/t7qA1HuDKVJnsf2ZhQP8p1dRF/cCYFOnOYYC
...
...
-----END RSA PRIVATE KEY-----

EDIT:

At last I solved this by

delete the public key in the server ending with same combination of user@host. I think paramiko sees some confusion there.
specify a user that exists on the remote machine in my connect(user=xxx). It is not documented and implicit; I am John in my machine, so the key generated has name of John@myhost in the key, but when I login into the remote Linux machine without a user named John, I have to connect(user=root/amy/any valid user on remote) to login in... I thought I don't have to specify a username in remote when I login with keypairs.

WesleyBlancoYuan on 12 Mar 2018

It happens again when I try to login to another machine, which hasn't my key pair. This is on purpose to see how paramiko works.

I think the logic of judging if a key is valid is too simply that the exception I get does not indicates the real problem. Please fix.

SSHException: not a valid OPENSSH private key file

WesleyBlancoYuan on 27 Mar 2018

Same problem as described above.
My key starts with:
-----BEGIN RSA PRIVATE KEY-----
Using 2.4.0.

Update:

OK - In my case, I found it isn't failing to identify the key - although it looks like that in the error message. It turns out it is silently trying using RSA. However, when it fails it tries a bunch of other stuff, including sshing with an id_dsa file floating around, which has an RSA key inside. So if you get this message it's probably because it's failed and is trying other things. In my case there were two mistakes: 1. I had the wrong username (passing the wrong thing to --ssh-username)

I was using the wrong servers! To generate the dask_servers.txt file I now run:
aws ec2 --region us-east-2 describe-instances --query "Reservations[*].Instances[*].PublicIpAddress" --output=text > dask_servers.txt
once I'd fixed those it sshed successfully (but other errors happened...).

dask-ssh --hostfile dask_servers.txt --ssh-private-key ~/.ssh/research_ohio.pem --ssh-username ubuntu

Notes:
specifically in transport.py and auth_handler.py where this 'issue' lies. Specifically, auth_handler.py maybe should be allowed to raise an exception which isn't caught? So we can see that it's tried and failed authenticating?

lionfish0 on 20 Apr 2018

I solved my issue, which I will share:

changed my code from:
k = paramiko.RSAKey.from_private_key_file(keyfile)

to:
k = paramiko.DSSKey.from_private_key_file(keyfile)

I ran into this 3 years ago, once, _and since forgot all about it.._ and I had to revisit this code today.
FORTUNATELY, back then, I had the foresight to enter this life-saving comment:

    # Paramiko uses different classes for different key-types.
    # Hint: TESTENV uses RSAKey and TESTENV2 uses DSSKey. Paramiko won't autodetect.

("won't autodetect" is not at all a judgement. Paramiko's one of my favorite Python libraries. Thanks Jeff (and all)!).

sprive on 17 Jan 2019

Hi folks, I know this is now closed but just in case anyone else comes here searching for an answer that might not be one of the above, had a similar error message as well.

How was the key generated? @Ghost0817 used ssh-keygen command in his instance. If done without specifying the output as a PEM file, it turns out that Paramiko can raise issues. More info here:

https://github.com/paramiko/paramiko/issues/1313

Hope this helps!

nickconIO on 20 Mar 2019

👍6 🎉3

I'm running into this issue with Paramiko 2.4.2.
Using the Paramiko SSHClient class, which actually _does_ try to auto-detect the key type, the key handling is failing on a valid key (works fine with command line ssh -i ...).

Relevant lines:

        if not two_factor:
            for key_filename in key_filenames:
                for pkey_class in (RSAKey, DSSKey, ECDSAKey, Ed25519Key): 
                    try:
                        key = self._key_from_filepath(
                            key_filename, pkey_class, passphrase
                        )

eepstein on 19 Apr 2019

This is a problem with recent macOS, whose OpenSSL ssh-keygen has changed the default format. Now a generated SSH key starts with:

-----BEGIN OPENSSH PRIVATE KEY-----

instead of the supported

-----BEGIN RSA PRIVATE KEY-----

To generate a supported key, use the following command:

ssh-keygen -t rsa -b 4096 -C "[email protected]" -m PEM

hyperknot on 15 May 2019

👍39 🎉9 😕3 ❤1

FWIW a lot of the recent comments in here should be addressed whenever I find new-feature-work time for #387, or may be handled in other feature tickets like #602.

bitprophet on 31 May 2019

This is a problem with recent macOS, whose OpenSSL ssh-keygen has changed the default format. Now a generated SSH key starts with:
-----BEGIN OPENSSH PRIVATE KEY-----
instead of the supported
-----BEGIN RSA PRIVATE KEY-----
To generate a supported key, use the following command:
ssh-keygen -t rsa -b 4096 -C "[email protected]" -m PEM

Not just a MacOS issue: Happened to me, using Git Bash in Windows.
Just specifying output format as PEM (-m PEM) solved the problem!
Thanks!