Lnd: Domain name and refresh external IP

[jochemin]

Background

I have a LND node running 24*7. It has a no-ip hostname associated with his external ip, in the configuration file (lnd.conf) I have this:

externalip=jocheminlnd1.ddns.net

If I launch a getinfo it returns the node public IP.

This public IP changes from time to time, lnd node continues working but active channels and peers drop and I have to stop client and launch again.

Is there a way to refer to the domain name and not the IP? or refresh the ip automatically on change?

Your environment

• LND 0.4.1
• Linux srv1 4.4.0-119-generic #143-Ubuntu
• Bitcoin Core 0.16

Steps to reproduce

Start your node with domain name in config file (lnd.conf) and change external IP.

Expected behaviour

Refer to the node by his domain name to avoid fails on public ip changing.

Actual behaviour

This public IP changes from time to time, lnd node continues working but active channels and peers drop and I have to stop client and launch again.

[vegardengen]

In the weekend, I experimented with creating an "lncli addexternalip" command, so that you could wrap this in script. I have it almost working, the only problem is that I don't get a new node announcement sent out - hopefully it's just a small misunderstanding from my side. Once that is done, I will continue with making an "lncli deleteexternalip" command. This would help some, right?

Other than that - would it be possible to add a lncli gracefulrestart - where wallet is kept open, and everything else is reinitialized? If there is no fundamental things stopping this, I could put it on my working list for future contributions.

[Roasbeef]

This public IP changes from time to time, lnd node continues working but active channels and peers drop and I have to stop client and launch again.

We'll automatically establish outbound connections to any peers that we have channels open with.

In the weekend, I experimented with creating an "lncli addexternalip" command, so that you could wrap this in script.

I don't think this is the best solution, as you'll need to have a secondary script just to monitor IP address changes and then tell lnd to use the latest one. Instead, if you're attempting to run a _routing_ node it's desired that you have a _stable_ IP address, otherwise peers will have trouble connecting out to you in the case that they don't even store any new NodeAnnouncements.

[vegardengen]

Well. There are different versions of stable. I am running on a dynamic IP that only rarely changes. But in the offshoot that it should change, I'm for example running a script to update DNS.

I think it's a valid use case. But I can code it myself and test it :)

[vegardengen]

Well - since you can have a domain name in the config file (resolved at startup time) - couldn't this also be solved if we created an "lncli reload" command?

lncli reload would do everything an lncli stop does, but stopping short of closing the wallet - and then opening/initializing everything again, except not having to reopen the wallet?

[jochemin]

I think it would be better to refer to domain name if it's in the config file, but reload option is interesting.

[vegardengen]

Ok. I have had an "addexternalip" command in test on my node for a few days. Not at all sure if it's preferable, I think perhaps lncli reload makes more sense, to be able to update all parameters without reopening wallets.

A couple of days ago, I restarted my node and added an ip adress. For a day and a half, the old nodeannouncement was used, not a new one - even though I generated and signed a new one, with a new timestamp.

Then, this happens. A new channel is opened, gossiping starts, and a new nodeannouncement is generated and gossiped. Will look more into this, unless we find the approach of an lncli reload better :)

2018-04-11 10:26:58.877 [DBG] PEER: Sending NodeAnnouncement(node=020f0fc08e5ca751f5fbd803b06236cde8e87035cb651893950d91c69e6a079957, update_time=2018-04-09 21:29:21 +0000 UTC) to 13.73.26.236:52168
2018-04-11 10:26:58.885 [DBG] PEER: Sending NodeAnnouncement(node=020f0fc08e5ca751f5fbd803b06236cde8e87035cb651893950d91c69e6a079957, update_time=2018-04-09 21:29:21 +0000 UTC) to 13.73.26.236:52168
2018-04-11 10:28:36.373 [DBG] PEER: Sending NodeAnnouncement(node=020f0fc08e5ca751f5fbd803b06236cde8e87035cb651893950d91c69e6a079957, update_time=2018-04-09 21:29:21 +0000 UTC) to 13.73.26.236:52168
2018-04-11 10:28:36.385 [DBG] PEER: Sending NodeAnnouncement(node=020f0fc08e5ca751f5fbd803b06236cde8e87035cb651893950d91c69e6a079957, update_time=2018-04-09 21:29:21 +0000 UTC) to 13.73.26.236:52168
2018-04-11 10:42:17.864 [INF] CRTR: New channel discovered! Link connects 020f0fc08e5ca751f5fbd803b06236cde8e87035cb651893950d91c69e6a079957 and 027b7e4cc67c0cfe8f314a0f5e682363afb56c7d6473b9ae70a36da5a52560b7d9
with ChannelPoint(80d50c665b2f9b0139926a3201169225099d2b882f51b596e6a809ef119f2c48:0): chan_id=1420915369256419328, capacity=0.009 BTC
2018-04-11 11:17:01.028 [INF] CRTR: Updated vertex data for node=020f0fc08e5ca751f5fbd803b06236cde8e87035cb651893950d91c69e6a079957
2018-04-11 11:17:24.332 [DBG] PEER: Sending NodeAnnouncement(node=020f0fc08e5ca751f5fbd803b06236cde8e87035cb651893950d91c69e6a079957, update_time=2018-04-11 11:17:01 +0000 UTC) to 71.127.210.173:55250
2018-04-11 11:17:24.332 [DBG] PEER: Sending NodeAnnouncement(node=020f0fc08e5ca751f5fbd803b06236cde8e87035cb651893950d91c69e6a079957, update_time=2018-04-11 11:17:01 +0000 UTC) to 103.102.44.207:36684
2018-04-11 11:17:24.332 [DBG] PEER: Sending NodeAnnouncement(node=020f0fc08e5ca751f5fbd803b06236cde8e87035cb651893950d91c69e6a079957, update_time=2018-04-11 11:17:01 +0000 UTC) to 138.197.121.209:9735
2018-04-11 11:17:24.333 [DBG] PEER: Sending NodeAnnouncement(node=020f0fc08e5ca751f5fbd803b06236cde8e87035cb651893950d91c69e6a079957, update_time=2018-04-11 11:17:01 +0000 UTC) to 100.38.3.152:61149
2018-04-11 11:17:24.333 [DBG] PEER: Sending NodeAnnouncement(node=020f0fc08e5ca751f5fbd803b06236cde8e87035cb651893950d91c69e6a079957, update_time=2018-04-11 11:17:01 +0000 UTC) to 87.79.255.30:60526
2018-04-11 11:17:24.333 [DBG] PEER: Sending NodeAnnouncement(node=020f0fc08e5ca751f5fbd803b06236cde8e87035cb651893950d91c69e6a079957, update_time=2018-04-11 11:17:01 +0000 UTC) to 86.61.67.183:9735
2018-04-11 11:17:24.333 [DBG] PEER: Sending NodeAnnouncement(node=020f0fc08e5ca751f5fbd803b06236cde8e87035cb651893950d91c69e6a079957, update_time=2018-04-11 11:17:01 +0000 UTC) to 13.73.26.236:52168
2018-04-11 11:17:24.333 [DBG] PEER: Sending NodeAnnouncement(node=020f0fc08e5ca751f5fbd803b06236cde8e87035cb651893950d91c69e6a079957, update_time=2018-04-11 11:17:01 +0000 UTC) to 34.250.234.192:9735
2018-04-11 11:17:24.333 [DBG] PEER: Sending NodeAnnouncement(node=020f0fc08e5ca751f5fbd803b06236cde8e87035cb651893950d91c69e6a079957, update_time=2018-04-11 11:17:01 +0000 UTC) to 67.166.50.158:43354
2018-04-11 11:17:24.333 [DBG] PEER: Sending NodeAnnouncement(node=020f0fc08e5ca751f5fbd803b06236cde8e87035cb651893950d91c69e6a079957, update_time=2018-04-11 11:17:01 +0000 UTC) to 107.220.128.247:40806
2018-04-11 11:17:24.333 [DBG] PEER: Sending NodeAnnouncement(node=020f0fc08e5ca751f5fbd803b06236cde8e87035cb651893950d91c69e6a079957, update_time=2018-04-11 11:17:01 +0000 UTC) to 81.18.224.62:52744
2018-04-11 11:17:24.333 [DBG] PEER: Sending NodeAnnouncement(node=020f0fc08e5ca751f5fbd803b06236cde8e87035cb651893950d91c69e6a079957, update_time=2018-04-11 11:17:01 +0000 UTC) to 58.6.87.18:50468
2018-04-11 11:17:24.333 [DBG] PEER: Sending NodeAnnouncement(node=020f0fc08e5ca751f5fbd803b06236cde8e87035cb651893950d91c69e6a079957, update_time=2018-04-11 11:17:01 +0000 UTC) to 180.181.208.42:9735

[Stadicus]

I can confirm that changing external ip addresses are the single most asked questdion in combination with my RaspiBolt guide. @Roasbeef, I don't think the requirement to have a fixed ip address to be able to route payments should just be assumed. At least in Europe fixed ip addresses are only available with business ISP contracts that are quite expensive. I think this goes directly against the goal to have a decentralized mesh network by raising the barriers to entry for non-business participants. And at least in my case, I do not have a fixed ip, but the address changes every few weeks, if not months.

I think the lncli reload command (restarting the service without locking up the wallet) would be a good solution. Monitoring ip address changes with a script is trivial and if a routing node is connected with a dyndns fqdn, a graceful restart would solve these issues.

On the other hand, the need to use a dyndns fqdn may not suit everybody, and as the whole Lightning specification works on bare ip addresses, this abstraction layer may not be welcomed. This would be more in favour in manually changing the ip (eg. with lncli updateexternalip) that is announced to the network.

Thanks @vegardengen for working on this after our Slack discussion. I would appreciate a broader discourse how to best enable this valid requirement.

[vegardengen]

@Stadicus I have most of the addexternalip stuff coded, only thing is that it's not currently gossiped before something else triggers the gossiping. I think I might code it as addexternalip and deleteexternalip, and not updateexternalip, though. Only thing, I guess, is that you don't want to trigger gossiping twice if you are doing an add/delete combination.

[vegardengen]

I also belief there is proof in working code. I am fully aware that @Roasbeef doesn't think it makes sense. My generating working code is an argument, not a protest. And being an LND newbie, it's useful training anyhow, whether or not it is merged in the end.

[Stadicus]

@vegardengen Is it the intent to allow multiple externalip addresses? This is what it seems to imply by add... and delete.... ip addresses and this is why I phrased it as update.... But any functionality to make LND usable with dynamic ips is fine with me.

Let me know if I can help in any way, eg. testing or documenting. I'm not much of a programmer, unfortunately... :-)

[vegardengen]

Multiple externalip is already supported - so I think add/delete is a more general solution than update. deleteexternalip is also a way of stopping your node from getting new connections, should you want that for a period.

[Stadicus]

Ah, did't consider that. It makes more sense now. 👍

[Roasbeef]

I guess what I'm trying to say, is that most of this effort can be pushed out to scripts that run along side lnd. We already have a signmessage command, so a script can re-create its _own_ NodeAnnouncement, and then have that be pushed out to lnd in an automated fashion. We can't be adding new RPC's for every little contrived feature (like adding a new _IP_) as otherwise, we'll just unnecessarily bloat the RPC interface with things that can instead be easily run along side lnd as a set of scripts.

For things like reloading the config manually using an RPC, that itself has several complications, as there're items that can _only_ be set at start up, such as which backend you're using, etc.

[Roasbeef]

At least in Europe fixed ip addresses are only available with business ISP contracts that are quite expensive.
And at least in my case, I do not have a fixed ip, but the address changes every few weeks, if not months.

Time to IPv6? Heh, only partially kidding...

I certainly see the use case, so seems the option is: restarting every few weeks, or adding a command to allow scripts to update the current set of advertised IPs for a node.

[Stadicus]

An external script to do that would definitely help, as a restart of lnd always forces unlocking the wallet again.

Any pointers how that could work? I coud work on the details given the knowledge where to start and how a good design could work on an abstract level.

[jochemin]

I think a good aproach would be 'announce' option. I assume a domain name in the config file.

Launching lncli announce will refresh only the ip in the "uris". The user could make a script to track his public ip and if changes send the announce option automatically.

[vegardengen]

What about creating a command:

updatenodeinfo

Here you can:

--addexternalip (to add)
--deleteexternalip (delete)
--setexternalip (to set the (list of) ip-adresses, disregarding the ones there before
--setalias

Would it also make sense to create an --announce true/false options, where you specify whether or not you want to gossip it now or wait till next change of node announcement?

@jochemin approach could also work, not sure what I prefer.

[vegardengen]

@Roasbeef I value your opinion here. My choice is between:

1) Adding separate addexternalip, deleteexternalip (and possibly later setalias)
2) Adding an updatenodeannouncement that will change all of these, with parameters :)

It's also a question whether or not we should gossip immediately or just wait until next time nodeannouncement is going to change anyhow. The first will actually make it easier to spam the network with unwanted info.

[halseth]

Related PR: https://github.com/lightningnetwork/lnd/pull/518

[Roasbeef]

So we're reviving #518 now. The new version of the PR will have a background goroutine that'll detect IP address changes and send out new announcements to update the network for reachability.

I think this addresses all the desired features detailed in this issue? The process would be automatic, and just all the setup guides would direct users to set the appropriate flag. This should work for the general case, but not for anything beyond a double NAT.

[Roasbeef]

Fixed by #1109.

[Chrozayis]

I am continuing to have issues with a dynamic IP over UPnP. The broadcast URI does not update unless I do a full restart of my node.

[egzumer]

One problem is that when router is restarted, then the UPnP ports are lost and not reestablished by lnd.
Second problem is that it looks like changed IP isn't reannounced after it changes. All block explorers show the old IP address unless lnd is restarted.