Lisk-sdk: Investigate sequences behaviour on 1.0.0-beta.7.3 branch

[MaciejBaj]

Expected behavior

The behaviour of sequences needs to be further investigated, possible issue fixed,

Actual behavior

During Alphanet QA round of 1.0.0-beta.7.3 memory leak was gone. WS sockets were not increasing. The 1.0.0-beta.7 release was held - some nodes were stack on a random height.

No syncing / receiving transactions / applying a block activity on a master process was visible after. The only activity on the stack nodes was kept on updating peers and broadcasting.

2018-05-09 17:20:21 993: [inf] 2018-05-09 17:20:21 | Releasing enqueued broadcasts
2018-05-09 17:20:21 994: [inf] 2018-05-09 17:20:21 | Queue empty
2018-05-09 17:20:22 037: [dbg] 2018-05-09 17:20:22 | Updated peer 206.189.92.184:5000 - {"broadhash":"4ca3711b229114db7bf5a1930f4d991e69c370625a70755862b1b806cd8817cb","height":1162}
2018-05-09 17:20:26 994: [inf] 2018-05-09 17:20:26 | Releasing enqueued broadcasts
2018-05-09 17:20:26 995: [inf] 2018-05-09 17:20:26 | Queue empty
2018-05-09 17:20:27 531: [dbg] 2018-05-09 17:20:27 | Updated peer 167.99.103.73:5000 - {"broadhash":"4ca3711b229114db7bf5a1930f4d991e69c370625a70755862b1b806cd8817cb","height":1162}
2018-05-09 17:20:29 792: [dbg] 2018-05-09 17:20:29 | Updated peer 165.227.99.83:5000 - {"broadhash":"4ca3711b229114db7bf5a1930f4d991e69c370625a70755862b1b806cd8817cb","height":1162}
2018-05-09 17:20:31 995: [inf] 2018-05-09 17:20:31 | Releasing enqueued broadcasts
2018-05-09 17:20:31 996: [inf] 2018-05-09 17:20:31 | Queue empty
2018-05-09 17:20:34 515: [dbg] 2018-05-09 17:20:34 | Updated peer 165.227.219.215:5000 - {"broadhash":"4ca3711b229114db7bf5a1930f4d991e69c370625a70755862b1b806cd8817cb","height":1162}
2018-05-09 17:20:36 997: [inf] 2018-05-09 17:20:36 | Releasing enqueued broadcasts
2018-05-09 17:20:36 998: [inf] 2018-05-09 17:20:36 | Queue empty
2018-05-09 17:20:40 462: [dbg] 2018-05-09 17:20:40 | Updated peer 146.185.164.56:5000 - {"broadhash":"4ca3711b229114db7bf5a1930f4d991e69c370625a70755862b1b806cd8817cb","height":1162}
2018-05-09 17:20:41 072: [dbg] 2018-05-09 17:20:41 | Updated peer 104.236.207.213:5000 - {"broadhash":"f8511aa78a6894b8e2a9135fc76662058a0461d5515719d8cbb1f9a7f2208c8e","height":1163}

Blocking tasks in sequences would explain it.

Steps to reproduce

QA of 1.0.0-beta.7.3

Which version(s) does this affect? (Environment, OS, etc...)

1.0.0-beta.7.3, potentially the previous versions

[shuse2]

It seems callback is not called when we try to use non-existing socket.
https://github.com/LiskHQ/lisk/blob/development/api/ws/rpc/connect.js#L148-L150