Lnd: LND fails to start when bitcoin RPC credentials have a "!" in them

[CubicEarth]

Tested with:
Bitcoind 0.16
LND 0.4
Ubuntu 16.04

Issue: If the Bitcoin rpcuser or rpcpassword contain the ' ! ' character, LND will exit with a 401 status code after the wallet has been unlocked. The issue exits if the ' ! ' is present at the end, or in the middle of the string, i.e.

rcpuser=iamcool!!
or
rpcuser=iamcool!!yes

The relevant log section:

2018-03-22 13:55:55.982 [INF] LTND: Waiting for wallet encryption password. Use `lncli create` to create wallet, or `lncli unlock` to unlock already created wallet.
2018-03-22 13:56:09.260 [INF] LNWL: Opened wallet
2018-03-22 13:56:09.340 [INF] LTND: Primary chain is set to: bitcoin
2018-03-22 13:56:09.340 [INF] LTND: Initializing bitcoind backed fee estimator
2018-03-22 13:56:10.495 [INF] LNWL: Opened wallet
unable to start wallet: status code: 401, response: ""
unable to create chain control: status code: 401, response: ""
status code: 401, response: ""

[bretton]

I had this with the "@" character, but it worked fine with just letters and numbers

I imagine same would apply for "#"

[Lldenaurois]

I’ll take this on as my first issue-ish

I tested with

bitcoind 0.16.0 (regtest mode)
lnd 0.4.0 (commit 18e6705d97840bfe4040a8a373ba40bc6a1b4b59)
Mac OS X

EDIT: I was able to reproduce this issue. There seem to be a few issues with how lnd talks to bitcoind when passing the rpcuser and rpcpass.

1. On my computer (Mac OS X), lnd is able to connect to bitcoind if the rpcuser or rpcpass contain a "!" anywhere
2. On my computer (Mac OS X), lnd is unable to connect to bitcoind if the rpcuser or rpcpass contain a "#" anywhere
3. If the rpcuser or rpcpass which is passed to lnd is different than the rpcuser which is passed to bitcoind, then lnd cannot start wallet, cannot create chain control and returns a 401 response.

unable to start wallet: status code: 401, response: ""
unable to create chain control: status code: 401, response: ""
status code: 401, response: ""

I will look into how errors are handled when the wallet starts and chain control is created to see whether we can guard against scenario "3" above.

I believe that for scenario "1" (which I was not able to reproduce) or "2", the username and password certificate (is it a certificate) is being computed or passed incorrectly. I believe it would make sense to attempt to connect to the rpc server manually using an rpcuser that includes the "#"

Reproduction Scenario (Mac OS X)

bitcoin.conf
rpcuser=kek!kek
rpcpass=kek!
regtest=1
txindex=1
server=1
daemon=1
zmqpubrawblock=tcp://127.0.0.1:28332
zmqpubrawtx=tcp://127.0.0.1:28332

lnd.conf
regtest=1
txindex=1
server=1
daemon=1
zmqpubrawblock=tcp://127.0.0.1:28332
zmqpubrawtx=tcp://127.0.0.1:28332

Reproduction Scenario:

When I pass the rpcuser and rpcpass to lnd via the cli, I get a bash error:

$ lnd --bitcoind.rpcuser=kek!!kek --bitcoind.rpcpass=kek!
-bash: !: event not found

But if I add it to the lnd.conf file, it works

lnd.conf
rpcuser=kek!@#kek
rpcpass=kek!
regtest=1
txindex=1
server=1
daemon=1
zmqpubrawblock=tcp://127.0.0.1:28332
zmqpubrawtx=tcp://127.0.0.1:28332

If I drop the "!" from both rpcuser and rpc pass, I get the 401.
$ lnd --bitcoind.rpcuser=kekkek --bitcoind.rpcpass=kek
2018-03-24 11:11:51.634 [INF] LTND: Version 0.4.0-beta
2018-03-24 11:11:51.634 [INF] LTND: Active chain: Bitcoin (network=regtest)
2018-03-24 11:11:51.634 [INF] CHDB: Checking for schema update: latest_version=0, db_version=0
2018-03-24 11:11:51.645 [INF] LTND: Primary chain is set to: bitcoin
2018-03-24 11:11:52.567 [INF] LNWL: Opened wallet
unable to start wallet: status code: 401, response: ""
unable to create chain control: status code: 401, response: ""
status code: 401, response: ""

If I keep the password correct (including the "!")
$ lnd --bitcoind.rpcuser=kekkek --bitcoind.rpcpass=kek!
2018-03-24 11:12:52.698 [INF] LTND: Version 0.4.0-beta
2018-03-24 11:12:52.698 [INF] LTND: Active chain: Bitcoin (network=regtest)
2018-03-24 11:12:52.698 [INF] CHDB: Checking for schema update: latest_version=0, db_version=0
2018-03-24 11:12:52.710 [INF] LTND: Primary chain is set to: bitcoin
2018-03-24 11:12:53.656 [INF] LNWL: Opened wallet
unable to start wallet: status code: 401, response: ""
unable to create chain control: status code: 401, response: ""
status code: 401, response: ""

Interestingly, however:

If I change the rpcuser to:
rpcuser=kek!#kek

it will no longer allow me to connect.

[Lldenaurois]

Ok, I looked into this a little further.

I am unable to connect to bitcoind in regtest mode via curl when my rpc username or password contains a "#", but I am able to connect if my username or password contains a "!" or "@".

I wonder if we should simply add a precondition or warning to lnd that lets you know that "!" or "#" is not suitable for a password? I need to figure out why for @CubicEarth it was "!", while for me it's "#".

Maybe we should move this issue to the bitcoind repo, since it is an issue with rpcuser and rpcpass in bitcoind.

bitcoin.conf
rpcuser=kek#kek
rpcpassword=kek#
regtest=1
txindex=1
server=1
daemon=1
zmqpubrawblock=tcp://127.0.0.1:28332
zmqpubrawtx=tcp://127.0.0.1:28332

~/netrc
machine localhost
login kek#kek
password kek#

$ curl --netrc-file ~/netrc --data-binary '{"jsonrpc": "1.0", "id":"curltest", "method": "getnetworkinfo", "params": [] }' localhost:18443

(no response, simply times out with nothing returned)

if I change the rpcuser to kek!kek and rpcpass to kek! in bitcoin.conf, and update my ~/netrc file, I get the following:

$ curl --netrc-file ~/netrc --data-binary '{"jsonrpc": "1.0", "id":"curltest", "method": "getnetworkinfo", "params": [] }' localhost:18443

{"result":{"version":160000,"subversion":"/Satoshi:0.16.0/","protocolversion":70015,"localservices":"000000000000040d","localrelay":true,"timeoffset":0,"networkactive":true,"connections":0,"networks":[{"name":"ipv4","limited":false,"reachable":true,"proxy":"","proxy_randomize_credentials":false},{"name":"ipv6","limited":false,"reachable":true,"proxy":"","proxy_randomize_credentials":false},{"name":"onion","limited":true,"reachable":false,"proxy":"","proxy_randomize_credentials":false}],"relayfee":0.00001000,"incrementalfee":0.00001000,"localaddresses":[],"warnings":""},"error":null,"id":"curltest"}

[bretton]

I've run through the following tests, however I'm not getting the error I had before, only an error on (#) which I think is treated as a comment modifier, and (") when it occurred twice as this seems to mess up bash.

baseline

Result: works

bitcoin.conf

rpcuser=test
rpcpassword=test

lnd.conf

[Bitcoind]
bitcoind.rpcuser=test
bitcoind.rpcpass=test

exclamation point variant 1

Result: works

bitcoin.conf

rpcuser=test
rpcpassword=te!st

lnd.conf

[Bitcoind]
bitcoind.rpcuser=test
bitcoind.rpcpass=te!st

exclamation point variant 2

Result: works

bitcoin.conf

rpcuser=te!st
rpcpassword=te!st

lnd.conf

[Bitcoind]
bitcoind.rpcuser=te!st
bitcoind.rpcpass=te!st

exclamation point variant 3

Result: works

bitcoin.conf

rpcuser=!test
rpcpassword=!test

lnd.conf

[Bitcoind]
bitcoind.rpcuser=!test
bitcoind.rpcpass=!test

hash mark

Result: does NOT work, can't unlock wallet, and bitcoin logs show

ThreadRPCServer incorrect password attempt from [::1]:60734

bitcoin.conf

rpcuser=test
rpcpassword=te#st

lnd.conf

[Bitcoind]
bitcoind.rpcuser=test
bitcoind.rpcpass=te#st

atsign variant 1

Result: works

bitcoin.conf

rpcuser=test
rpcpassword=te@st

lnd.conf

[Bitcoind]
bitcoind.rpcuser=test
bitcoind.rpcpass=te@st

atsign variant 2

Result: works

bitcoin.conf

rpcuser=@test
rpcpassword=@test

lnd.conf

[Bitcoind]
bitcoind.rpcuser=@test
bitcoind.rpcpass=@test

quotes variant 1

Result: works

bitcoin.conf

rpcuser=test
rpcpassword=te"st

lnd.conf

[Bitcoind]
bitcoind.rpcuser=test
bitcoind.rpcpass=te"st

quotes variant 2

Result: fails, lnd won't start

bitcoin.conf

rpcuser="test
rpcpassword="test

lnd.conf

[Bitcoind]
bitcoind.rpcuser="test
bitcoind.rpcpass="test

[Lldenaurois]

what’s interesting is that it’s an issue for bitcoind. You cannot connect to bitcoind via rpc if you have a “#” in either rpcpass or rpcuser.

[CubicEarth]

I discovered the error when testing with credentials that contained double '!!'.

[Lldenaurois]

@CubicEarth I copy psted your exact credentials and it works on my computer, no issue!

I only have issues with # and “

[bretton]

I am able to use "!!" for user/pass, lnd starts fine, no rpc errors.

Note, I did have an issue previously and simply dropped any punctuation from credentials. However with the more exhaustive testing today it seems to only exist with # and "

[Lldenaurois]

@bretton: The issue is with bitcoind, not lnd. Check my comment regarding curl commands to bitcoind in regtest modes and please try out on your side. You should not be able to connect to bitcoind via rpc (curl) if your rpcuser has a # in it. Same for rpcpass

[bretton]

@Lldenaurois yep I've already confirmed a problem with # in credentials :)

that probably is bitcoind, the problem with " twice over seems to be something in bash

[CubicEarth]

Today I am not getting the error with the !! . I tested it several times before posting, so that is strange.

Makes sense that the underlying issue is in bitcoind, although also surprising it hasn't been caught and fixed by now.

[Lldenaurois]

@CubicEarth: I think you just mixed up the credentials. If you pass the wrong rocuser or rpcpass you get the same 401 in lnd

[CubicEarth]

@Lldenaurois - I was able to recreate it with rpcuser=test!!test and rpcpassword=test!!test , although it seems like the issue is that when my terminal encounters the '!!', it recalls the previous command. So not an LND problem.

[Lldenaurois]

@CubicEarth: Just add the password with double exclamation points to your lnd.conf and it shpuld work!

You can’t pass it in bash via cli call

[CubicEarth]

So the problems discovered have either been related to my unrealistic expectations of what bash can do, or to underlying problems with how Bitcoin handles certain characters, such as # or " . Should we close this issue?

[Lldenaurois]

I would move this issue to bitcoin repo

[CubicEarth]

I am not familiar with github etiquette and protocol.
Are you suggesting this whole thread be 'reassigned' to the bitcoin repo as-is?
Or should it be closed here, and then a new issue raised over there?