Xud: Orders data structures

[moshababo]

To support internal matching, ownOrders and peerOrders are merged in MatchingEngine priority queues, but we still need to support an access to the orders separately. For ownOrders it's for the GET_ORDERS packet response, and for peerOrders it's for subscribePeerOrders gRPC call.

First we need to decide on whether or not we need them sorted (by price), when being served separately. I don't see a requirement for that right now, so if someone does, say so.

In addition, these lists are currently being managed inside of OrderBook, which is separated from the main work in MatchingEngine. I think it will be better to have everything being managed under one place, so that would be in inside of MatchingEngine.

In addition, @sangaman suggested to make the priority queue a secondary index, so that the orders would be saved elsewhere, to allow easy updates (via orderId, the pk). To do so, the queue will still require to have the order's price & createdAt, because that's what we're sorting on. So I don't see a benefit from splitting the order object. We can have the same object sitting both on the priority queue and on a separate list ({ [orderId: string]: Order }), and use the later for fast pk lookups, when doing read/update. add/delete will require operations on both.

EDIT: this would also allow easy update of an order quantity, which doesn't affect the order priority. At the moment it's implemented in OrderBook.removePeerOrder by removing the order from the priority queue, editing it, and adding it again.

[kilrau]

First we need to decide on whether or not we need them sorted (by price), when being served separately. I don't see a requirement for that right now, so if someone does, say so.

At the moment we have 3 places to

1. Orderbook , peerOrders (key-value pairs)
2. Orderbook, ownOrders (key-value pairs)
3. MatchingEngine (FastPriorityQueue, Sell, Buy, Sell)

GET_ORDERS is publicly exposed. Shouldn't be sorted.

subscribePeerOrders - grpc to exchange, tbd: include ownOrders too. @moshababo @sangaman
Leaning towards: would be useful to be sorted in the beginning, but not critical since new orders come in as single events and need to be continuously sorted by exchange all the time anyways.

In addition, these lists are currently being managed inside of OrderBook, which is separated from the main work in MatchingEngine. I think it will be better to have everything being managed under one place, so that would be in inside of MatchingEngine.

Agree, drop orderbook class, move 1. Orderbook, peerOrders 2. Orderbook, ownOrders into MatchingEnglines class.

[sangaman]

We can have the same object sitting both on the priority queue and on a separate list ({ [orderId: string]: Order }), and use the later for fast pk lookups, when doing read/update. add/delete will require operations on both.

This seems like the best approach.

[sangaman]

Also if we merge the OrderBook and MatchingEngine classes together I feel like we should come up with a more descriptive name, particularly since we'll still need the lists even if we're not in matching mode. I'm not sure what the best name is right now though, and it might depend on the specifics of how we implement things.

[moshababo]

I think we should keep both OrderBook & MatchingEngine. The later will contain all logic for orders management, per a specific pair. It should be easily tested with unit tests. Yes, It won't be focused solely on matching, but matching involves with orders management anyway.
OrderBook is the higher layer, which interacts with the p2p layer (listening, broadcasting), emitting events, creating invoices, etc. It shouldn't contain orders management logic.

[kilrau]

So the MatchingEngine class will still be necessary when xud runs in non-matching mode? @moshababo

[moshababo]

We still haven't thought through the implementation details of that mode (figuring what still need to be managed inside xud). We might be able to cut off MatchingEngine entirely. But even if we don't, it's not a big deal.

[kilrau]

We still haven't thought through the implementation details of that mode (figuring what still need to be managed inside xud). We might be able to cut off MatchingEngine entirely. But even if we don't, it's not a big deal.

Can we continue this? I think we should tackle this in alpha.2, maybe even in alpha.1 if we get clarity soon @moshababo @sangaman