Electrs: can't shutdown gracefully when bitcoin-rpc is down

[sangaman]

I stopped bitcoind and afterwards tried to shutdown electrs with SIGINT. However, electrs got stuck on CLOSING RPC CONNECTIONS and after 20 minutes or so I had to force kill the process.

WARN - failed to connect daemon at 127.0.0.1:8332: Connection refused (os error 111)
TRACE - RPC Done
DEBUG - [127.0.0.1:60646] shutting down connection
INFO - [127.0.0.1:60646] disconnected peer
WARN - failed to connect daemon at 127.0.0.1:8332: Connection refused (os error 111)
WARN - failed to connect daemon at 127.0.0.1:8332: Connection refused (os error 111)
WARN - failed to connect daemon at 127.0.0.1:8332: Connection refused (os error 111)
WARN - failed to connect daemon at 127.0.0.1:8332: Connection refused (os error 111)
WARN - failed to connect daemon at 127.0.0.1:8332: Connection refused (os error 111)
WARN - failed to connect daemon at 127.0.0.1:8332: Connection refused (os error 111)
WARN - failed to connect daemon at 127.0.0.1:8332: Connection refused (os error 111)
WARN - failed to connect daemon at 127.0.0.1:8332: Connection refused (os error 111)
WARN - failed to connect daemon at 127.0.0.1:8332: Connection refused (os error 111)
WARN - failed to connect daemon at 127.0.0.1:8332: Connection refused (os error 111)
WARN - failed to connect daemon at 127.0.0.1:8332: Connection refused (os error 111)
^CTRACE - stop accepting new RPCs
TRACE - closing RPC connections

[romanz]

Good catch, thanks!

[romanz]

I failed to reproduce the issue (at e446693c4c96840199178b65b5c63e3766108c99) :(

My steps:

• Started bitcoind, electrs and electrum.
• Stopped bitcoind, waited ~14 seconds.
• Stopped electrum, waited ~4 seconds.
• Send SIGINT to electrs (via Ctrl+C) --> seems to exit correctly.

2018-12-01T15:02:29.848+02:00 - WARN - reconnecting to bitcoind: failed to read cookie from "/home/roman/.bitcoin/.cookie"
2018-12-01T15:02:32.849+02:00 - WARN - failed to connect daemon at 127.0.0.1:8332: Connection refused (os error 111)
2018-12-01T15:02:35.850+02:00 - WARN - failed to connect daemon at 127.0.0.1:8332: Connection refused (os error 111)
2018-12-01T15:02:38.850+02:00 - WARN - failed to connect daemon at 127.0.0.1:8332: Connection refused (os error 111)
2018-12-01T15:02:41.851+02:00 - WARN - failed to connect daemon at 127.0.0.1:8332: Connection refused (os error 111)
2018-12-01T15:02:43.621+02:00 - TRACE - RPC Done
2018-12-01T15:02:43.621+02:00 - DEBUG - [127.0.0.1:55742] shutting down connection
2018-12-01T15:02:43.622+02:00 - INFO - [127.0.0.1:55742] disconnected peer
2018-12-01T15:02:44.851+02:00 - WARN - failed to connect daemon at 127.0.0.1:8332: Connection refused (os error 111)
^C2018-12-01T15:02:47.741+02:00 - TRACE - stop accepting new RPCs
2018-12-01T15:02:47.741+02:00 - TRACE - closing RPC connections
2018-12-01T15:02:47.741+02:00 - TRACE - RPC connections are closed
2018-12-01T15:02:47.743+02:00 - TRACE - RPC server is stopped
2018-12-01T15:02:47.745+02:00 - TRACE - closing DB at "/home/roman/.electrs/mainnet"
2018-12-01T15:02:47.766+02:00 - ERROR - server failed: Error: Iterrupted by SIGINT

[romanz]

Could you please re-produce this issue with 7061c53d973da97aeb116a5135a8c1ca27064fca, to understand better where it gets stuck?

[romanz]

Does this issue still happen?

[sangaman]

I haven't seen it happen again but I haven't been shutting down as often. I can probably try to reproduce it this weekend, I'll update this issue when I have time to try. Thanks.

[sangaman]

Just a heads up, I shutdown electrs for the first time in weeks or months, and I'm again stuck on closing RPC connections. I'm going to update to master now, and I will try to remember to shut down via sig int in a few days just to see how it goes and will update this issue.