I am exposing the json-rpc api to a http server plugin
It generally seems to work great, however listtransactions and listfunds seems to get their wires crossed..
I make a command like so (via http POST request):
request: {"jsonrpc":"2.0","id":"1","method":"listtransactions","params":[]}
which returns:
json: {
id = 1;
jsonrpc = "2.0";
result = {
channels = (
{
"amount_msat" = 100000000msat;
"channel_sat" = 100000;
"channel_total_sat" = 100000;
connected = 1;
"funding_output" = 0;
"funding_txid" = 89bb7b71dab90bf5c36f460f9cb00b72a645236929321a9f83024f7bd1be23a0;
"our_amount_msat" = 100000000msat;
"peer_id" = 039b1717db1193eb332d3c0bfdcce90a6aab60efa478b60963d3b406a8fc45134a;
"short_channel_id" = 1803776x9x0;
state = "CHANNELD_NORMAL";
},
{
"amount_msat" = 100000000msat;
"channel_sat" = 100000;
"channel_total_sat" = 100000;
connected = 0;
"funding_output" = 0;
"funding_txid" = 3cbd0682de3d7370bf17111ccb220ea7b583375ab1169d1ec1de88ec7e1e24b6;
"our_amount_msat" = 100000000msat;
"peer_id" = 03fce2a20393a65b9d6cab5425f4cd33ddc621ade458efd69d652917e2b5eaf59c;
"short_channel_id" = 1803783x30x0;
state = ONCHAIN;
},
{
"amount_msat" = 100000000msat;
"channel_sat" = 99990;
"channel_total_sat" = 100000;
connected = 1;
"funding_output" = 0;
"funding_txid" = ba7cf2118c63496b10ab89abe6533f1302846ba42eadfe93cc41607e721f09a1;
"our_amount_msat" = 99990000msat;
"peer_id" = 03fce2a20393a65b9d6cab5425f4cd33ddc621ade458efd69d652917e2b5eaf59c;
"short_channel_id" = 1803856x5x0;
state = "CHANNELD_NORMAL";
},
{
"amount_msat" = 1000000000msat;
"channel_sat" = 1000000;
"channel_total_sat" = 1000000;
connected = 1;
"funding_output" = 0;
"funding_txid" = e425d67ce116c015be8151da1b97f0fdb73023280e70a1bbeda69822334214f9;
"our_amount_msat" = 1000000000msat;
"peer_id" = 03724f73513718c8ed09f96faec8b9d295f61140be30d43fda09edf886161ff9d8;
"short_channel_id" = 1803794x1x0;
state = "CHANNELD_AWAITING_LOCKIN";
}
);
outputs = (
{
address = tb1qd89chgrvp0373ljz7hk4yphftg5d2xzuxld8lq;
"amount_msat" = 27699051000msat;
blockheight = 1803856;
output = 1;
reserved = 0;
scriptpubkey = 001469cb8ba06c0be3e8fe42f5ed5206e95a28d5185c;
status = confirmed;
txid = ba7cf2118c63496b10ab89abe6533f1302846ba42eadfe93cc41607e721f09a1;
value = 27699051;
}
);
};
}
However the exact same command without any changes will also return:
{
"transactions": [
{
"hash": "25e2f592be76c7714445b37cc2bbe9cbd59fc940e209517624195ced7039913a",
"rawtx": "020000000001020a29eb6ea256ecef3c83aa370134ebe3af312a82ba5d73b97822a676f7b1539c0100000000feffffffd1c96a89c07b9d9dd71c99c01f851bf722a9fcef4aa0902076301cd37377fc710100000000feffffff01f37fba0100000000160014a3cb12b1f56e5b377b28948147a462d7ce1e4cdf02473044022060a87a4af2a41944ea7a6dd8f320aa0f5d5be7394f0eb26e3979c5e932bc68bc022013a1ad02e2794e93f2e8634a6f47c5f5e61182c41f42a6bb949d46e56331b53c012102ee5bd1becd65e703b876b3dd121cc3bb067f886535d955d6622566ba8965214c02473044022039ac963514179e4456990b9d3fe2cb6623a23c1f674951e2051d13f4dc0df1f70220760528e7f7ed6660cc72409726e28e8aabbcba3d9945897b3c30651cdc6a92ab0121033616cc3f524613210882d5a02edc89f7de46e444026b2d5d3e6ec40758c0132eb7851b00",
"blockheight": 1803706,
"txindex": 13,
"locktime": 1803703,
"version": 2,
"inputs": [
{
"txid": "9c53b1f776a62278b9735dba822a31afe3eb340137aa833cefec56a26eeb290a",
"index": 1,
"sequence": 4294967294
},
{
"txid": "71fc7773d31c30762090a04aeffca922f71b851fc0991cd79d9d7bc0896ac9d1",
"index": 1,
"sequence": 4294967294
}
],
"outputs": [
{
"index": 0,
"satoshis": "28999667000msat",
"scriptPubKey": "0014a3cb12b1f56e5b377b28948147a462d7ce1e4cdf"
}
]
},
{
"hash": "89bb7b71dab90bf5c36f460f9cb00b72a645236929321a9f83024f7bd1be23a0",
"rawtx": "02000000013a913970ed5c1924765109e240c99fd5cbe9bbc27cb3454471c776be92f5e2250000000000feffffff02a0860100000000002200203ee6a25e370c2bed17cc33c7d8bb49f492b81fd80d8f96f8a12afadcb0c0d484b9f8b8010000000016001434a8d7fed808ee6c5d3b7b932dc9a3fd39017ac2ff851b00",
"blockheight": 1803776,
"txindex": 9,
"locktime": 1803775,
"version": 2,
"inputs": [
{
"txid": "25e2f592be76c7714445b37cc2bbe9cbd59fc940e209517624195ced7039913a",
"index": 0,
"sequence": 4294967294
}
],
"outputs": [
{
"index": 0,
"satoshis": "100000000msat",
"scriptPubKey": "00203ee6a25e370c2bed17cc33c7d8bb49f492b81fd80d8f96f8a12afadcb0c0d484"
},
{
"index": 1,
"satoshis": "28899513000msat",
"scriptPubKey": "001434a8d7fed808ee6c5d3b7b932dc9a3fd39017ac2"
}
]
},
{
"hash": "3cbd0682de3d7370bf17111ccb220ea7b583375ab1169d1ec1de88ec7e1e24b6",
"rawtx": "0200000001a023bed17b4f02839f1a3229692345a6720bb09c0f466fc3f50bb9da717bbb890100000000feffffff02a08601000000000022002083730c1b307671190c88375ef4ad52a0817bf19d44fbf9ab2b22e29ab92656927f71b70100000000160014712ab8facea1567d0da8938c180e76863886f41404861b00",
"blockheight": 1803783,
"txindex": 30,
"locktime": 1803780,
"version": 2,
"inputs": [
{
"txid": "89bb7b71dab90bf5c36f460f9cb00b72a645236929321a9f83024f7bd1be23a0",
"index": 1,
"sequence": 4294967294
}
],
"outputs": [
{
"index": 0,
"satoshis": "100000000msat",
"scriptPubKey": "002083730c1b307671190c88375ef4ad52a0817bf19d44fbf9ab2b22e29ab9265692"
},
{
"index": 1,
"satoshis": "28799359000msat",
"scriptPubKey": "0014712ab8facea1567d0da8938c180e76863886f414"
}
]
},
{
"hash": "e425d67ce116c015be8151da1b97f0fdb73023280e70a1bbeda69822334214f9",
"rawtx": "0200000001b6241e7eec88dec11e9d16b15a3783b5a70e22cb1c1117bf70733dde8206bd3c0100000000feffffff0240420f00000000002200205734ca03c926ee1d296145409ca68af30100b80b70996d4e741e2a655a1cf008a52ea80100000000160014769ad91844455b891fd3b3e151dfbb76917136c50f861b00",
"blockheight": 1803794,
"txindex": 1,
"locktime": 1803791,
"version": 2,
"inputs": [
{
"txid": "3cbd0682de3d7370bf17111ccb220ea7b583375ab1169d1ec1de88ec7e1e24b6",
"index": 1,
"sequence": 4294967294
}
],
"outputs": [
{
"index": 0,
"satoshis": "1000000000msat",
"scriptPubKey": "00205734ca03c926ee1d296145409ca68af30100b80b70996d4e741e2a655a1cf008"
},
{
"index": 1,
"satoshis": "27799205000msat",
"scriptPubKey": "0014769ad91844455b891fd3b3e151dfbb76917136c5"
}
]
},
{
"hash": "8fcc8fa18004a42a3549f466b478d105ffbd12b9aa0184bf284ab845cd89d863",
"rawtx": "02000000000101b6241e7eec88dec11e9d16b15a3783b5a70e22cb1c1117bf70733dde8206bd3c00000000005acc728001e985010000000000220020e449986d97b370e382744e981f4a60319e16ef237549ab6f67ec1e5ed454b67404004730440220722f7494ae7de864579a38642fb210c472f8dfa4d805797fafb1c37a825256fc0220491d05484d6ff6d4df8c0b5a8c4a3e04607919f73edaaffd478efed8e4963afd0147304402204bc684adc245e53c0f14c155c05069270f1d3b4d8e4963dfa9c1eb115f547ae9022045ca382b2d9bad5b9e058128f02b79f4acdc4bc3025982cf9d25a32b773a0202014752210246b102a98d9028e4a28d2acf1aa9677569ef0a5e16453ee09620eea809d3b79e210399b3760628206985aaf942f81bbbc4b716bc00248f2667c18a1ceaad6e11bc4152ae875bc620",
"blockheight": 1803844,
"txindex": 8,
"locktime": 549870471,
"version": 2,
"inputs": [
{
"txid": "3cbd0682de3d7370bf17111ccb220ea7b583375ab1169d1ec1de88ec7e1e24b6",
"index": 0,
"sequence": 2155007066
}
],
"outputs": [
{
"index": 0,
"satoshis": "99817000msat",
"scriptPubKey": "0020e449986d97b370e382744e981f4a60319e16ef237549ab6f67ec1e5ed454b674"
}
]
},
{
"hash": "ba7cf2118c63496b10ab89abe6533f1302846ba42eadfe93cc41607e721f09a1",
"rawtx": "0200000001f91442332298a6edbba1700e282330b7fdf0971bda5181be15c016e17cd625e40100000000feffffff02a086010000000000220020f503c63afb6f953492b46c2227623c3aadee24b99faf7d12938880ad4e8304996ba7a6010000000016001469cb8ba06c0be3e8fe42f5ed5206e95a28d5185c4d861b00",
"blockheight": 1803856,
"txindex": 5,
"locktime": 1803853,
"version": 2,
"inputs": [
{
"txid": "e425d67ce116c015be8151da1b97f0fdb73023280e70a1bbeda69822334214f9",
"index": 1,
"sequence": 4294967294
}
],
"outputs": [
{
"index": 0,
"satoshis": "100000000msat",
"scriptPubKey": "0020f503c63afb6f953492b46c2227623c3aadee24b99faf7d12938880ad4e830499"
},
{
"index": 1,
"satoshis": "27699051000msat",
"scriptPubKey": "001469cb8ba06c0be3e8fe42f5ed5206e95a28d5185c"
}
]
}
]
}
Stranger still:
Sometimes making the command:
{"jsonrpc":"2.0","id":"1","method":"listfunds","params":[]}
returns:
transactions = (
{
blockheight = 1803706;
hash = 25e2f592be76c7714445b37cc2bbe9cbd59fc940e209517624195ced7039913a;
inputs = (
{
index = 1;
sequence = 4294967294;
txid = 9c53b1f776a62278b9735dba822a31afe3eb340137aa833cefec56a26eeb290a;
},
{
index = 1;
sequence = 4294967294;
txid = 71fc7773d31c30762090a04aeffca922f71b851fc0991cd79d9d7bc0896ac9d1;
}
);
locktime = 1803703;
outputs = (
{
index = 0;
satoshis = 28999667000msat;
scriptPubKey = 0014a3cb12b1f56e5b377b28948147a462d7ce1e4cdf;
}
);
rawtx = 020000000001020a29eb6ea256ecef3c83aa370134ebe3af312a82ba5d73b97822a676f7b1539c0100000000feffffffd1c96a89c07b9d9dd71c99c01f851bf722a9fcef4aa0902076301cd37377fc710100000000feffffff01f37fba0100000000160014a3cb12b1f56e5b377b28948147a462d7ce1e4cdf02473044022060a87a4af2a41944ea7a6dd8f320aa0f5d5be7394f0eb26e3979c5e932bc68bc022013a1ad02e2794e93f2e8634a6f47c5f5e61182c41f42a6bb949d46e56331b53c012102ee5bd1becd65e703b876b3dd121cc3bb067f886535d955d6622566ba8965214c02473044022039ac963514179e4456990b9d3fe2cb6623a23c1f674951e2051d13f4dc0df1f70220760528e7f7ed6660cc72409726e28e8aabbcba3d9945897b3c30651cdc6a92ab0121033616cc3f524613210882d5a02edc89f7de46e444026b2d5d3e6ec40758c0132eb7851b00;
txindex = 13;
version = 2;
},
{
blockheight = 1803776;
hash = 89bb7b71dab90bf5c36f460f9cb00b72a645236929321a9f83024f7bd1be23a0;
inputs = (
{
index = 0;
sequence = 4294967294;
txid = 25e2f592be76c7714445b37cc2bbe9cbd59fc940e209517624195ced7039913a;
}
);
locktime = 1803775;
outputs = (
{
index = 0;
satoshis = 100000000msat;
scriptPubKey = 00203ee6a25e370c2bed17cc33c7d8bb49f492b81fd80d8f96f8a12afadcb0c0d484;
},
{
index = 1;
satoshis = 28899513000msat;
scriptPubKey = 001434a8d7fed808ee6c5d3b7b932dc9a3fd39017ac2;
}
);
rawtx = 02000000013a913970ed5c1924765109e240c99fd5cbe9bbc27cb3454471c776be92f5e2250000000000feffffff02a0860100000000002200203ee6a25e370c2bed17cc33c7d8bb49f492b81fd80d8f96f8a12afadcb0c0d484b9f8b8010000000016001434a8d7fed808ee6c5d3b7b932dc9a3fd39017ac2ff851b00;
txindex = 9;
version = 2;
},
{
blockheight = 1803783;
hash = 3cbd0682de3d7370bf17111ccb220ea7b583375ab1169d1ec1de88ec7e1e24b6;
inputs = (
{
index = 1;
sequence = 4294967294;
txid = 89bb7b71dab90bf5c36f460f9cb00b72a645236929321a9f83024f7bd1be23a0;
}
);
locktime = 1803780;
outputs = (
{
index = 0;
satoshis = 100000000msat;
scriptPubKey = 002083730c1b307671190c88375ef4ad52a0817bf19d44fbf9ab2b22e29ab9265692;
},
{
index = 1;
satoshis = 28799359000msat;
scriptPubKey = 0014712ab8facea1567d0da8938c180e76863886f414;
}
);
rawtx = 0200000001a023bed17b4f02839f1a3229692345a6720bb09c0f466fc3f50bb9da717bbb890100000000feffffff02a08601000000000022002083730c1b307671190c88375ef4ad52a0817bf19d44fbf9ab2b22e29ab92656927f71b70100000000160014712ab8facea1567d0da8938c180e76863886f41404861b00;
txindex = 30;
version = 2;
},
{
blockheight = 1803794;
hash = e425d67ce116c015be8151da1b97f0fdb73023280e70a1bbeda69822334214f9;
inputs = (
{
index = 1;
sequence = 4294967294;
txid = 3cbd0682de3d7370bf17111ccb220ea7b583375ab1169d1ec1de88ec7e1e24b6;
}
);
locktime = 1803791;
outputs = (
{
index = 0;
satoshis = 1000000000msat;
scriptPubKey = 00205734ca03c926ee1d296145409ca68af30100b80b70996d4e741e2a655a1cf008;
},
{
index = 1;
satoshis = 27799205000msat;
scriptPubKey = 0014769ad91844455b891fd3b3e151dfbb76917136c5;
}
);
rawtx = 0200000001b6241e7eec88dec11e9d16b15a3783b5a70e22cb1c1117bf70733dde8206bd3c0100000000feffffff0240420f00000000002200205734ca03c926ee1d296145409ca68af30100b80b70996d4e741e2a655a1cf008a52ea80100000000160014769ad91844455b891fd3b3e151dfbb76917136c50f861b00;
txindex = 1;
version = 2;
},
{
blockheight = 1803844;
hash = 8fcc8fa18004a42a3549f466b478d105ffbd12b9aa0184bf284ab845cd89d863;
inputs = (
{
index = 0;
sequence = 2155007066;
txid = 3cbd0682de3d7370bf17111ccb220ea7b583375ab1169d1ec1de88ec7e1e24b6;
}
);
locktime = 549870471;
outputs = (
{
index = 0;
satoshis = 99817000msat;
scriptPubKey = 0020e449986d97b370e382744e981f4a60319e16ef237549ab6f67ec1e5ed454b674;
}
);
rawtx = 02000000000101b6241e7eec88dec11e9d16b15a3783b5a70e22cb1c1117bf70733dde8206bd3c00000000005acc728001e985010000000000220020e449986d97b370e382744e981f4a60319e16ef237549ab6f67ec1e5ed454b67404004730440220722f7494ae7de864579a38642fb210c472f8dfa4d805797fafb1c37a825256fc0220491d05484d6ff6d4df8c0b5a8c4a3e04607919f73edaaffd478efed8e4963afd0147304402204bc684adc245e53c0f14c155c05069270f1d3b4d8e4963dfa9c1eb115f547ae9022045ca382b2d9bad5b9e058128f02b79f4acdc4bc3025982cf9d25a32b773a0202014752210246b102a98d9028e4a28d2acf1aa9677569ef0a5e16453ee09620eea809d3b79e210399b3760628206985aaf942f81bbbc4b716bc00248f2667c18a1ceaad6e11bc4152ae875bc620;
txindex = 8;
version = 2;
},
{
blockheight = 1803856;
hash = ba7cf2118c63496b10ab89abe6533f1302846ba42eadfe93cc41607e721f09a1;
inputs = (
{
index = 1;
sequence = 4294967294;
txid = e425d67ce116c015be8151da1b97f0fdb73023280e70a1bbeda69822334214f9;
}
);
locktime = 1803853;
outputs = (
{
index = 0;
satoshis = 100000000msat;
scriptPubKey = 0020f503c63afb6f953492b46c2227623c3aadee24b99faf7d12938880ad4e830499;
},
{
index = 1;
satoshis = 27699051000msat;
scriptPubKey = 001469cb8ba06c0be3e8fe42f5ed5206e95a28d5185c;
}
);
rawtx = 0200000001f91442332298a6edbba1700e282330b7fdf0971bda5181be15c016e17cd625e40100000000feffffff02a086010000000000220020f503c63afb6f953492b46c2227623c3aadee24b99faf7d12938880ad4e8304996ba7a6010000000016001469cb8ba06c0be3e8fe42f5ed5206e95a28d5185c4d861b00;
txindex = 5;
version = 2;
}
)
Seems like the JSON-RPC api is getting its wires crossed?
Does not seem to be specific to those two calls, it seems to happen anytime there are consecutive rpc calls made. Really strange as it also seems to be completely random.
@Fonta1n3 you receive this result with the http server plugin? or with c-lightning? I think with the plugin, right?
@Fonta1n3 you receive this result with the http server plugin? or with c-lightning? I think with the plugin, right?
Right. Will raise there.
I think that it is a plugin error when it build the response from the Http request, not it a c-lightning problem. We can not do-nothings here
Are you not changing the id each time? Should you not use a UUID for the id?
(If you were using a persistent connection then you could just use an incrementing id, but HTTP is not a persistent connection, it is a connect-request-respond-disconnect protocol, so you probably need a UUID in the id field I think, otherwise the http plugin might have trouble putting back which id belongs to which HTTP request)
Are you not changing the
ideach time? Should you not use a UUID for theid?(If you were using a persistent connection then you could just use an incrementing id, but HTTP is not a persistent connection, it is a connect-request-respond-disconnect protocol, so you probably need a UUID in the
idfield I think, otherwise the http plugin might have trouble putting back which id belongs to which HTTP request)
TIL :) Thank you! I am used to json-rpc 1.0 where i always had curltest hardcoded as the id.
@Fonta1n3, maybe you can find a solution to your problem with another project, I know a javascript rest implementation or another rust rest implementation.
Of course, if you like to play with Java there is also my implementation of rest plugin but at the moment is only a draft.
Maybe this can help, I know that there are the same thinks of the plugins that are you using
@vincenzopalazzo I will certainly check them out! Thanks for sharing.
I am able to shed more light on the issue now, I am going to reopen until I solve this even though it may well have nothing to do with c-lightning.
I thought @ZmnSCPxj point about the ID was the culprit so I now am using a UUID for each call but the issue persists. The plugin is stateless so it does not make sense that it is the culprit if it does not store anything in memory.
The issue I am finding only arises after a rpc command times out, I am using a hidden service so the commands time out from time to time. Is it at all possible that a commands response gets stuck in c-lightning memory if it is never delivered?
To be very clear what happens is (for example) listfunds gets called and returns a successful response, then listtransactions gets called and times out, I then make a newaddr call and the response from listtransactions gets returned. From the point of one call timing out things seem to go pear shaped, if nothing times out then everything just works as it should.
One other important thing, rebooting c-lightning (and yes of course the plugins) seems to resolve the issue, until a request times out.
Depends on how the plugin constructs the JSON-RPC interface to the lightningd, and how it re-associates results from lightningd to separate HTTP requests. Far more likely the plugin is at fault, as it is newer code compared to our JSON-RPC code.
@Fonta1n3 If you have time we can try to reproduce this bug with another plugin, such as my lightning-rest plugin because it uses a server framework inside and I don't think to have this problem but at the moment is only a draft.
If you want, you can DM me on #c-lightning freenode or you can DM with any contact in my Github page
P.S: I'm curious to understand because this bug happened
@vincenzopalazzo The authors of the plugin have pushed an update that seems to resolve the issue :) I would be very happy to test these other plugins, I will be updating Fully Noded soon to be c-lightning capable, but a user must setup a plug in for http server and must expose the http port to a Tor V3 HS in order for it to work, it will certainly be good for me to offer users a multitude of plugins to choose from. Just in case you are interested you can see the latest commit here https://github.com/Start9Labs/c-lightning-http-plugin to see what the fix was.
Thanks to you and @ZmnSCPxj for being so responsive to this even though it was not a c-lightning issue.
@Fonta1n3 amazing :-) happy to know that you resolved your bug. I noted that the bug was inside the RPC plugin wrapper, as @ZmnSCPxj suggested.
@Fonta1n3, maybe you can find a solution to your problem with another project, I know a javascript rest implementation or another rust rest implementation.
Of course, if you like to play with Java there is also my implementation of rest plugin but at the moment is only a draft.
Maybe this can help, I know that there are the same thinks of the plugins that are you using
These seem great but I think only yours (and the one I am using) is offering a "pure" inerface with lightning-cli which is what I need. Am I correct to assume that is what yours does @vincenzopalazzo ?
Hi @Fonta1n3,
Sorry about the delay, I lost the notify.
Do you mean about the rest plugin? lightning-rest
If yes, the idea of this plugin is make the access to lightningd with http request. In other words, the interface is the same as lightning-cli
Is possible to run the server and stop the server with lightning-cli.
At the moment this interface is very a draft because I'm developing the java RPC wrapper
P.S: feel free to open an issue if you need some some function not implemented yet