Tskit: edge table overflows without being detected, I think?

[bhaller]

A SLiM user is running a model that involves 2000 individuals, with a genome consisting of 2000 unlinked "chunks" with a recombination rate of 0.5 between them (to make them unlinked). This means that each new offspring genome generated involves recording ~1000 crossovers (half the 0.5 points generate a crossover on average), so for 2000 diploid individuals there are ~4,000,000 edges recorded in the edge table in each generation. They're playing with the simplification interval, and found that if they don't simplify for 1000 generations they get an error. This seems to me to make sense, because 4,000,000 x 1000 is pushing 2^32; they're probably just overflowing the maximum size of the edge table. So fine, they should simplify more often, I have no problem with that.

The bug here, at first blush, is that the error message is poor: "add_edge: Unknown error". Looking at the code involved, however, I think something more problematic is occurring under the hood. First of all, I'm not immediately seeing a check for overflowing the maximum size of the table, as dictated by the 32-bit range of table_size_t; perhaps I'm missing that check somewhere. But second, I think it is not even getting that far. Once it expands past 2^31, it looks like the code in edge_table_add_row() does "ret = (edge_id_t)self->num_rows" and then returns ret. This casts num_rows, which is a table_size_t and thus a uint32_t, to an edge_id_t which is an int_32_t. When the table size is > 2^31, it therefore returns a negative number, which is interpreted as an error code, but of course it is essentially a random number and often does not correspond to any defined error code – thus "unknown error".

So I guess there are a couple of things going on here; the table needs to detect that it has been asked to resize beyond its limit, and throw an error about that right up front. And two, the limit needs to be 2^31, not 2^32, if you're going to return the number of rows as a signed int and use negative numbers as error codes. Obviously it's unfortunate to impose a 2^31 limit just for that reason; but if edge_id_t is int_32_t then perhaps that limit is necessary for other reasons as well.

This issue is of no particular urgency to me. It's an unusual model that is clearly pushing the limits, and I don't expect other people to hit this problem often. I've told them that they need to simplify more often, and explained to them that they're just overflowing the edge table; they are fine. But eventually this does seem like a problem that ought to be fixed, since SLiM users like to do crazy things and overflow the tables. :->

[jeromekelleher]

I haven't dealt with overflows on the tables at all @bhaller, so you're definitely right this is a problem. Thanks for the bug report, and apologies for the sloppiness. This will be fixed for the next C release.

[jeromekelleher]

This should be fixed and release in C_0.99.2 @bhaller.