Lightning: Feature Request: RPC endpoint for Watch Tower (Simple)

[NicolasDorier]

Ideally, I would like clightning to have very simple watch tower capabilities.

The use case I have in mind has quite simple UX:

Merchant A knows Merchant B:

Merchant A share his watch tower endpoint (URI exposing a REST API made by BTCPAY) to Merchant B.

Merchant B can then configure his store to send watch information after every lightning payment received to the store. (Merchant B don't have to care to send watch information when he spends the money)

BTCPay would just relay this information to the underlying clightning instance.

Note that my case is quite simple: Merchant A knows Merchant B beforehand so no need of "discovery protocol". There is also no need of P2P level protocol at clightning level, as BTCPay is the one exposing the REST endpoints. Only things needed is a RPC method at clightning level to do this.

[ZmnSCPxj]

Hi @NicolasDorier, sorry but I cannot visualize exactly what you want as interface. Do you need some serialization of a specific channel state and RPC calls to get current channel state, and override that state? Or something else entirely?

[NicolasDorier]

Sorry I don't know a lot about the internal implementation of lightning.

The problem I want to solve is that right now, merchants are taking significant risks with lightning. If they get DDoS an attacker might be able to close the channel with an invalidated state.

I want to provide them a way to outsource channel monitoring.

In BTCPay, merchant knows each other, and can outsource such service to each other, I am not asking for any particular P2P protocol for watch towers. Just a RPC call where the merchant can tell to another merchant "If you see this old state on chain, broadcast this transaction to sweep the channel, so justice can be served".

It can be a simple monitor hash tx, where if the merchant see the preimage of hash on the blockchain, then he will broadcast tx. Though I think you may have a better way of monitoring channels in lightning.

[ZmnSCPxj]

Okay, that seems a little more designable.

Note however that currently our blockchain monitoring seems a little underoptimized: #1337.

Also ping @rustyrussell as I am unaware of status of plans for a "real" watchtower. I understand how merchants would want to have some assurance of channel justice even though a "real" watchtower system has not completed at LN yet.

Now, first some things:

• In LN, a commitment transaction held by party A has two branches: a CSV+sigA branch, and a combined (sigA+sigB) branch which happens to have a single priv and pubkey pair (do not ask me how, I never got around to studying the math). The party A revokes its old commitment by providing sigA to party B. Hashes are not used in the justice branch.
• What we monitor is spending of the funding transaction. When we see the funding transaction spent when we know we have not broadcast our commitment transaction or a cooperative bilateral close, then we assume it is the other party commitment transaction and try to get the justice branch of the contract.
• We store revocation keys using a "shachain" structure (do not ask me how that looks like, I never got around to studying the math). The shachain is O(1) storage for an indefinite number of revocation keys.
• Every transfer through a channel involves 2 updates: 1st to transfer funds to an HTLC, 2nd to transfer funds from HTLC to the destination. So we need to provide two revocation keys to the watchtower for each transfer through a channel.

A rough draft would be something like:

getshachain channelid [prev_shachain] - Get the current shachain representation for the current state of the given channel ID. Also provide funding_txid and funding_outnum. If prev_shachain is specified and the given prev_shachain matches the actual current shachain state, then block until a change in shachain state occurs.

monitorshachain funding_txid funding_outnum shachain address - Monitor the given funding outpoint. If its funding transaction gets spent, try a revocation key from the given shachain structure and send it to the given onchain address. If an existing monitor for that funding outpoint exists, overwrite the shachain and address.

Aside from getting the shachain, I think monitoring a funding transaction can be done in a separate project, not necessarily c-lightning. So the most minimal thing we need to provide is getshachain from lightningd.

[NicolasDorier]

I would be happy to implement the watch tower at NBXplorer level actually. Looking the chain in search for transaction matching features is already coded. If I can get the minimal information from clightning I need to be able to track, it should be pretty easy for my explorer.

It makes even more sense because BTCPay allow merchant to use their own lightning charge, so BTCPay can only create and monitor invoices but not steal any funds. The BTCPay host would be able to monitor channels on behalf of its merchants, this is pretty cool.

[ZmnSCPxj]

Okay, I will add the querying command for the current channel state.

[ZmnSCPxj]

@rustyrussell I would like to confirm my understanding of BOLT 3 and 5.

I am taking the point of view of the cheated node, as that is what a watchtower protects.

In order to make a justice transaction to handle my counterparty cheating transaction, I need:

1. per_commitment_secret that was revealed to me by the counterparty during revoke_and_ack message. This is stored in the shachain structure, which is a 64-bit minimum index, and up to 49 pairs of a 256-bit hash and 64-bit index. I identify the correct secret by getting the index in obscured form from the nLocktime and nSequence of the commitment transaction.
2. Both sides payment_basepoint. Aside from using my own payment_basepoint to claim the THEIR_UNILATERAL/TO_US output, the two basepoints are also used to unmask the obscured commitment index in the shachain, so that I can determine the commitment secret there.
3. My own side revocation_basepoint_secret. This is used in computing the revocation private key together with the per_commitment_point.
4. The other side delayed_payment_basepoint. This is used in computing the script for the THEIR_UNILATERAL/TO_THEM output, otherwise it would be impossible for me to claim this as this is a P2WSH.
5. Both sides htlc_basepoint. These are used in computing the scripts for HTLCs offered by us and by them, again these are P2WSH. I will use the revocation key to revoke these.

So aside from the shachain I need to provide these to the watchtower:

1. our_payment_basepoint
2. their_payment_basepoint
3. our_revocation_basepoint_secret
4. their_delayed_payment_basepoint
5. our_htlc_basepoint
6. their_htlc_basepoint
7. funding_txid and funding_outnum of the channel

[NicolasDorier]

So on my explorer I will need to find out if (funding_txid, funding_outnum) is spent.
Then from your command output, do you have a small process on how I can spend the output of transaction? (telling me the math)

[NicolasDorier]

Ah actually, can you also give me the scriptPubKey in hex of the funding as a convenience? This facilitate the scanning as I will not need to do an outpoint lookup in database. (I am already doing a lookup based on infered scriptPubKey from the scriptSig/witScript)

[ZmnSCPxj]

Then from your command output, do you have a small process on how I can spend the output of transaction? (telling me the math)

See BOLT 3 and BOLT 5. My understanding of the math is shaky and that is why I tried summoning @rustyrussell here: my explanation might make things worse by misinforming you!

There is also, the code in handle_their_cheat inside onchaind/onchain.c. Be warned that there is a lot of very very similar (BUT SUBTLY DIFFERENT) code in the various handle_* functions in the same file, do not get confused reading them. You could copy the handle_their_cheat function in a separate file to help against this.

AS I UNDERSTAND IT (please make sure to run this by a real mathematician), you get:

1. per_commitment_point = per_commitment_secret * G where G is the standard generator point for Bitcoin secp256k1.
2. Every _basepoint is computed similarly from every _secret, e.g. payment_basepoint = payment_secret * G. Basepoints are shared between both sides as they are public, but secrets are kept secret (i.e. our_revocation_basepoint_secret is given to the WatchTower and allows the WatchTower to collude with the other side to steal the funds).
3. The public keys in a specific commitment transaction use the per_commitment_point and the *_basepoint. For example, if you are claiming your output from their-side commitment transaction, you need the payment_pubkey for that commitment transaction, which is payment_pubkey = payment_basepoint + SHA256(per_commitment_point || payment_basepoint) * G, where SHA256 is the standard 256-bit SHA-2 hash function.
4. If you know the corresponding _secret you can get the corresponding _privkey for a specific _pubkey: The payment_privkey = payment_secret + SHA256(per_commitment_point || payment_basepoint).
5. Revocation public and private keys are derived slightly differently. To derive the revocation public key for the other side commitment, you use your own revocation_basepoint and the other side per_commitment_point as revocation_pubkey = SHA256(revocation_basepoint || per_commitment_point) * revocation_basepoint + SHA256(per_commitment_point || revocation_basepoint) * per_commitment_point.
6. When the other side reveals to you the private per_commitment_secret as part of revoke_and_ack, you can now compute the revocation_privkey as revocation_privkey = SHA256(revocation_basepoint || per_commitment_point) * revocation_secret + SHA256(per_commitment_point || revocation_basepoint) * per_commitment_secret.

Roughly, when you see the funding transaction spent:

1. Extract the commitment number. This is obscured in 24 bits of nLockTime and 24 bits of nSequence; the commitment number is obscured using the payment basepoints, see BOLT3 "Commitment Transaction".
2. Look up the commitment number in the shachain (algorithm is in BOLT3 under "Efficient Per-commitment Secret Storage", also see the ccan/ccan/shachain directory). The shachain is effectively a function index -> Maybe per_commitment_secret. Note that the shachain index starts from the max value 2^48 - 1, but the commitment number starts at 0. So make sure to convert correctly!
3. If the commitment number is not found in the shachain, or the resulting per_commitment_secret does not match the commitment transaction P2WPKH/P2WSH hashes, then it is either a valid commitment, our own commitment, or a bilateral close transaction. If so justice does not need to be served as it is valid. Either that or you have a bug in your code hahahaha.
4. For each output of the commitment, identify it and write a Bitcoin transaction input that claims it (so you make a txid-outnum reference and dummy scriptSig and witness data). There are four output types, see BOLT 5 for how each is handled in detail:
   
   
   
   1. Our-side main output. This is just a P2WPKH, so just make a normal P2WPKH claim. It should match the payment_pubkey for the given per_commitment_point of this commitment transaction and our own payment_basepoint.
   
   
   2. Their-side main output. This is a P2WSH paying out to a script involving revocation_pubkey (derived from our own revocation_basepoint and the per_commitment_point of the commitment) and their delayed_payment_pubkey (derived from their delayed_payment_basepoint and the per_commitment_point). You can claim this using a signature using the revocation_secret. See BOLT 3.
   
   
   3. Our offered HTLCs. This is a P2WSH involving revocation_pubkey, and our and their htlc_pubkey, and payment_preimage. See BOLT 3. You can claim this with the revocation_secret, and if the other side could claim it first the other side claim can also be revoked with the revocation_secret somehow.
   
   
   4. Their offered HTLCs. Again P2WSH, see BOLT 5 and BOLT 3 how to claim.
   
   
5. Attach an output that is all the inputs minus fees, going to an address you control (and in the case of WatchTower, to an output that the customer controls). Sign using the appropriate keys and broadcast!
6. Probably enable RBF in the justice transaction; as new blocks are found, lower the output (putting more to fees) until the justice transaction is confirmed or its inputs become claimed.

It might be useful to separate the claims of the HTLCs from the claims of the "main" outputs, i.e. multiple justice transactions, not sure though, wait for @rustyrussell to comment.

Ah actually, can you also give me the scriptPubKey in hex of the funding as a convenience? This facilitate the scanning as I will not need to do an outpoint lookup in database. (I am already doing a lookup based on infered scriptPubKey from the scriptSig/witScript)

Lemme see how easy it is; in principle funding_txid and funding_outnum is enough.

Would it not be in principle easier using funding_txid and funding_outnum, however? When a new block comes in, scan each txin for the funding_txid and funding_outnum? Or are you using an SPV interface like Electrum?

[NicolasDorier]

funding_txid and funding_outnum is enough. But my explorer is wired to query on scriptPubKeys, never outpoint. Adding one query for each outpoint for each transaction might make things too slow, so it would make my life way easier.

I could workaround this by using Bitcoin RPC to fetch the output. (I think)

[NicolasDorier]

Actually Bitcoin RPC does not seem to allow me to fetch an utxo (and thus the scriptPubKey I need) given the outpoint... I hate Bitcoin RPC (-_-)

Edit: Actually the REST interface of Bitcoin seems to expose this, but activating it is yet another setting I prefer not to have my user to manually set... so if you can give me the scriptPubKey you save me lot's of time and support...

[ZmnSCPxj]

Okay, to derive the scriptPubKey of the funding outpoint I need to make a 2-of-2 2 <pubkey1> <pubkey2> 2 OP_CHECKMULTISIG P2WSH, where <pubkey1> is the numerically lesser of the DER-encoded pubkeys of both sides. The local funding pubkey is derivable using derive_basepoints from the channel->seed, while the remote funding pubkey is stored in the channel->channel_info.remote_fundingkey.

[NicolasDorier]

@ZmnSCPxj I can reconstruct the scriptPubKey on my side if you prefer.

[ZmnSCPxj]

It is easy to get the scriptpubkey once the pubkeys are known, it was digging through our structures to find both pubkeys that was hard.

[ZmnSCPxj]

@NicolasDorier let us continue discussion here.

In #1381 @NicolasDorier described the use case more precisely:

Having a trusted watch tower seems to defeat the point. My goal was that a web of cooperating merchant agree to watch for each other without trusting each other.

As #1381 provides enough information for the WatchTower to write and sign the justice transactions, an obvious attack by WatchTower would be to direct the justice transaction output to itself and the counterparty in the channel. The counterparty publishes a revoked transaction, the WatchTower revokes it --- in favor of itself and the counterparty rather than the protected node.

The trustless alternative would be for the protected node to create and sign (multiple!) justice transactions for each revoked other side commitment transaction of each channel it has. This has the advantage that the WatchTower is incapable of rewriting the justice transaction. This has a lot of engineering changes in c-lightning:

1. We currently never save other side commitment transactions. With the signed-justice-transactions technique we need to save up to two commitment transactions of the other side: the current unrevoked commitment transaction, and the most recent one it made (and which will replace the current unrevoked commitment transaction when the current unrevoked is revoked).
2. We need to refactor the justice transaction writing from closingd/closing.c to something in common/.
3. We need to split up writing the justice transaction. Currently we rewrite the justice transaction depending on how the revoked commitment gets spent (e.g. if a revoked HTLC outpoint is spent by a HTLC-timeout we should change the justice transaction to spend the revocable HTLC-timeout instead). It would be unwieldy to provide a large number of justice transactions (we could have 483 HTLCs offered by one side plus another 483 HTLCs offered by the other, each of which could be directly revoked or revoked via HTLC-timeout/HTLC-success) so we should write multiple justice transactions for the main output and two for each HTLC on the channel, rather than the current technique of a single justice transaction claiming all of those.

Because of the rather large engineering changes I want to wait for comments from @rustyrussell and @cdecker.

[ZmnSCPxj]

Discussion on the mailinglist has been quite active.

The general consensus so far seems to be that the protected node will provide a (txid[:16], blob) pair, where txid[:16] means "the first 16 bytes of the txid". The "txid" here is the txid of the revoked commitment transaction.

The WatchTower process basically is that when a new block is mined, we take the txid of every transaction in the block. If the first 16 bytes of a txid matches one of the (txid[:16], blob) pairs being watched, then we use the last 16 bytes of the txid to decrypt the blob. If the blob is decrypted correctly (maybe it has some embedded HMAC or checksum to determine if the decryption was correct), then the blob contains signatures for one or more justice transactions that properly reclaim the revoked transaction. We use one justice transaction for the remote side main output, and one justice transaction for every HTLC, using the fact that the HTLC outpoints have two stages, both of which are revokable, and that timely behavior is needed only to claim the second stage (so the WatchTower does not need to be given a signature for a justice transaction that can claim the first stage, only the second CSV-locked stage).

This probably requires more discussion and thought, so if you do not mind, I would like to let this topic stew a little while before we proceed to something to implement as a first cut.

• We would prefer not to reveal the funding scriptpubkey if the WatchTower is trustless: we want to hide which channels the WatchTower is really monitoring unless the WatchTower actually needs to act. Hence this is keyed to the revoked commitment txid rather than funding scriptpubkey, sorry. The revoked commitment txid cannot reveal the funding outpoint unless the revoked commitment tx actually gets published, which we hope is a rare enough event.

[NicolasDorier]

I like the encrypted blob approach, it seems quite simple for the WatchTower.

[hayek-developer0]

Does c-lightning support watchtower now?

[ZmnSCPxj]

Not yet. It is likely we will imitate whatever lnd does.