Dvc: private key file is encrypted Error for SSH remote add

[shashankpr]

Please provide information about your setup
DVC version(i.e. dvc --version), Platform and method of installation (pip, homebrew, pkg Mac, exe (Windows), DEB(Linux), RPM(Linux))

DVC version: 0.90.2
Installed using: pip on Mac

Problem description

I have added an ssh remote and specified keyfile (.pem format) as given the DVC docs. Although, when I perform dvc add ssh://ubuntu@<aws ec2 ip>:/test/, I am getting the following error: ERROR: unexpected error - private key file is encrypted
Since it is an AWS instance managed by devops team in my workplace, I don't think there is a way that I can get or pass password for the keyfile. I wanted to know if there is any alternative way to resolve this ?

The full trace of error:

2020-03-21 17:27:52,580 DEBUG: PRAGMA user_version;
2020-03-21 17:27:52,581 DEBUG: fetched: [(3,)]
2020-03-21 17:27:52,581 DEBUG: CREATE TABLE IF NOT EXISTS state (inode INTEGER PRIMARY KEY, mtime TEXT NOT NULL, size TEXT NOT NULL, md5 TEXT NOT NULL, timestamp TEXT NOT NULL)
2020-03-21 17:27:52,581 DEBUG: CREATE TABLE IF NOT EXISTS state_info (count INTEGER)
2020-03-21 17:27:52,582 DEBUG: CREATE TABLE IF NOT EXISTS link_state (path TEXT PRIMARY KEY, inode INTEGER NOT NULL, mtime TEXT NOT NULL)
2020-03-21 17:27:52,582 DEBUG: INSERT OR IGNORE INTO state_info (count) SELECT 0 WHERE NOT EXISTS (SELECT * FROM state_info)
2020-03-21 17:27:52,582 DEBUG: PRAGMA user_version = 3;
2020-03-21 17:27:53,605 DEBUG: Establishing ssh connection with '<aws ec2 ip>' through port '22' as user 'ubuntu'
Adding...
2020-03-21 17:27:55,593 DEBUG: SELECT count from state_info WHERE rowid=?
2020-03-21 17:27:55,593 DEBUG: fetched: [(19,)]
2020-03-21 17:27:55,594 DEBUG: UPDATE state_info SET count = ? WHERE rowid = ?
2020-03-21 17:27:55,600 ERROR: unexpected error - private key file is encrypted
------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/shashank/anaconda3/lib/python3.7/site-packages/dvc/remote/pool.py", line 51, in get_connection
    return self._conns.popleft()
IndexError: pop from an empty deque

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/shashank/anaconda3/lib/python3.7/site-packages/dvc/main.py", line 50, in main
    ret = cmd.run()
  File "/Users/shashank/anaconda3/lib/python3.7/site-packages/dvc/command/add.py", line 20, in run
    fname=self.args.file,
  File "/Users/shashank/anaconda3/lib/python3.7/site-packages/dvc/repo/__init__.py", line 28, in wrapper
    ret = f(repo, *args, **kwargs)
  File "/Users/shashank/anaconda3/lib/python3.7/site-packages/dvc/repo/scm_context.py", line 4, in run
    result = method(repo, *args, **kw)
  File "/Users/shashank/anaconda3/lib/python3.7/site-packages/dvc/repo/add.py", line 80, in add
    stage.save()
  File "/Users/shashank/anaconda3/lib/python3.7/site-packages/dvc/stage.py", line 788, in save
    out.save()
  File "/Users/shashank/anaconda3/lib/python3.7/site-packages/dvc/output/base.py", line 202, in save
    if not self.exists:
  File "/Users/shashank/anaconda3/lib/python3.7/site-packages/dvc/output/base.py", line 160, in exists
    return self.remote.exists(self.path_info)
  File "/Users/shashank/anaconda3/lib/python3.7/site-packages/dvc/remote/ssh/__init__.py", line 142, in exists
    with self.ssh(path_info) as ssh:
  File "/Users/shashank/anaconda3/lib/python3.7/contextlib.py", line 112, in __enter__
    return next(self.gen)
  File "/Users/shashank/anaconda3/lib/python3.7/site-packages/dvc/remote/pool.py", line 11, in get_connection
    conn = pool.get_connection()
  File "/Users/shashank/anaconda3/lib/python3.7/site-packages/dvc/remote/pool.py", line 53, in get_connection
    return self._conn_func(*self._conn_args, **self._conn_kwargs)
  File "/Users/shashank/anaconda3/lib/python3.7/site-packages/dvc/remote/ssh/connection.py", line 47, in __init__
    self._ssh.connect(host, *args, **kwargs)
  File "/Users/shashank/anaconda3/lib/python3.7/site-packages/paramiko/client.py", line 446, in connect
    passphrase,
  File "/Users/shashank/anaconda3/lib/python3.7/site-packages/paramiko/client.py", line 764, in _auth
    raise saved_exception
  File "/Users/shashank/anaconda3/lib/python3.7/site-packages/paramiko/client.py", line 735, in _auth
    filename, pkey_class, passphrase
  File "/Users/shashank/anaconda3/lib/python3.7/site-packages/paramiko/client.py", line 586, in _key_from_filepath
    key = klass.from_private_key_file(key_path, password)
  File "/Users/shashank/anaconda3/lib/python3.7/site-packages/paramiko/pkey.py", line 235, in from_private_key_file
    key = cls(filename=filename, password=password)
  File "/Users/shashank/anaconda3/lib/python3.7/site-packages/paramiko/rsakey.py", line 55, in __init__
    self._from_private_key_file(filename, password)
  File "/Users/shashank/anaconda3/lib/python3.7/site-packages/paramiko/rsakey.py", line 175, in _from_private_key_file
    data = self._read_private_key_file("RSA", filename, password)
  File "/Users/shashank/anaconda3/lib/python3.7/site-packages/paramiko/pkey.py", line 308, in _read_private_key_file
    data = self._read_private_key(tag, f, password)
  File "/Users/shashank/anaconda3/lib/python3.7/site-packages/paramiko/pkey.py", line 337, in _read_private_key
    data = self._read_private_key_openssh(lines[start:end], password)
  File "/Users/shashank/anaconda3/lib/python3.7/site-packages/paramiko/pkey.py", line 439, in _read_private_key_openssh
    "private key file is encrypted"
paramiko.ssh_exception.PasswordRequiredException: private key file is encrypted

[efiop]

Hi @shashankpr !

Have you tried password or ask_password config options for your remote? See examples here https://dvc.org/doc/command-reference/remote/modify

[shashankpr]

Hi @efiop , I have tried password and ask_password option as well but the problem is that the remote host has been configured for keyfile login and I do not have access to the password (it is managed by a platform team). I am wondering if I can forward the Keyring through dvc remote configuration itself?

[efiop]

@shashankpr Oh, sorry, I'm unfamiliar with your setup and will need additional info. Do you have something special that handles your auth? Kerberos? Or something else?

[shashankpr]

@efiop , I was able to solve my issue by first running this:
ssh-add -K <pem file>
After this I wasn't asked for any password nor was there any encrypted error. Sorry for the trouble and the vague problem description. I will close the issue.