Lightning: [Bug] If bitcoind is not fully synched when clightning starts, clightning will start to scan

[NicolasDorier]

If bitcoind is not fully synched when clightning starts for the first time, it seems clightning is starting to scan blocks from the beginning of time.

See: https://github.com/btcpayserver/btcpayserver-docker/issues/18#issuecomment-411623713

Reported by @Sosthene and @ndeet , based on 0.6.

Workaround: Using a utilities waiting for bitcoind to be fully synched before starting lnd. (Using NBXplorer/NBXplorer.NodeWaiter in BTCPay)

[NicolasDorier]

At the very least clightning should not scan things before lightning was cool.
Better, clightning should not try to scan things while your bitcoind is synching and pruned.

BTCPay is using pruned node, so running clightning while bitcoind is not fully sync might end up badly as clightning might be too slow and bitcoind might prune blocks clightning is trying to scan.

By starting clightning only when bitcoind start, I make sure that clightning does not start scanning from long time ago.

Scanning a brand new wallet is only useful for getting the channels in the network, which is only useful when you want to send payment. For a merchant case, the most important is not to send but to receive, so they don't really care about the tradeoff.

[yaraaer]

If bitcoind is not fully synched when clightning starts for the first time, it seems clightning is starting to scan blocks from the beginning of time.

this sounds right. clightning has to walk the chain as well, fetching blocks from bitcoind as it goes. I'm not sure what the workaround accomplishes, you would still need to walk the chain even after bitcoind is fully synced. Perhaps I'm missing something...

it is not right. I restarted my server, but after reboot I didn't mount disk where bitcoind data were stored. I just started bitcoind, then clightning, not realizing that initial sync is running. I found out next day. I have stopped cln + bitcoind, erased useless bitcoind data, mounted storage, started bitcoin and get to synced state within few minutes - for bitcoind. but clightning did not recover so easily. it is syncing blocks from the begining and it is taking ages.
on top of that I have to stop/start cln time to time and do the backup of sqlite file. since it happens that file gets malformed during that sync (no idea why).
is there any work around to skip that long useless sync? I did try to manually replace "blocks" table, but that did not work for me. seems that there are some dependencies with other tables :-(
at least CLN should complain when being started with non-synced bitcoind.
probably that's what should be somehow modified? https://github.com/ElementsProject/lightning/blob/master/lightningd/chaintopology.c#L741

[NicolasDorier]

@yaraaer I ended up have a marker file file "bitcoind_fully_synched" that some middleware of mine is generating, and the script starting lightningd just sleeping until it sees this file before starting lightningd.

Hack, but never got any problem since then.

[darosior]

I think this is now handled correctly ?

[rustyrussell]

Nope, just got a report from Javier on Telegram. He ran bitcoind -reindex and it found corruption. lightningd started syncing from block 119,000 or so! Presumably he started it up before bitcoind was finished.

[JavierRSobrino]

Right, that was what happened to me today.

Nope, just got a report from Javier on Telegram. He ran bitcoind -reindex and it found corruption. lightningd started syncing from block 119,000 or so! Presumably he started it up before bitcoind was finished.

[cdecker]

This was fixed in faded9a9cf7e4fc01bcb9b03b0381e0ca738bfe1, and I verified that lightningd refuses to start with the following test:

def test_ibd(node_factory):
    l1 = node_factory.get_node(start=False)

    mock_getblockchaininfo = {
        "chain": "regtest",
        "blocks": 1000,
        "headers": 2000,
        "initialblockdownload": True,
    }

    l1.daemon.rpcproxy.mock_rpc('getblockchaininfo', mock_getblockchaininfo)
    l1.start()