Tskit: Provide TableCollection.dump( ) method

[petrelharp]

Currently to write out a table collection, you have to make it into a tree sequence first, which then just calls table_collection_dump() under the hood. Unless there's a good reason not, it'd be convenient sometimes to dump the table collection directly.

[petrelharp]

Also, .copy().

[petrelharp]

Also, a method to load them. Currently, if you have a set of tables that are illegal - for instance, because they lack a Population table - there's no easy way in python to load the tables, add the missing stuff, and then make a tree sequence out of them. I would have called this msprime.load_tables, but this is already taken.

[bhaller]

Loading tables certainly seems good. Dumping tables seems like it would sometimes be useful, but as I understand it the benefit of making a tree sequence and then dumping that is that various checks are performed, guaranteeing that the .trees file you write is well-formed. So if a way to dump tables directly is provided, perhaps the docs for it ought to at least warn that this means such checks are not done, and that dumping a ts may be safer...?

[petrelharp]

Hm, that's a good point. I guess the basic question here is: do we want users to be able to work directly with illegal tables, ever? (I think: yes; users won't be able to load the illegal tables into tree sequences anyhow, so no big deal...)

[bhaller]

(I think: yes; users won't be able to load the illegal tables into tree sequences anyhow, so no big deal...)

I think yes, but I also think it is worthwhile to warn users about the risks of doing so. As for the "no big deal" point, they might not be able to crash msprime with bad tables, but they might well be able to write them out to a .trees file and then crash some other program that is not entirely robust in its reading in of .trees files. I'm not advocating preventing them from doing so; I'm just saying that when they manage to paint themselves into a corner this way, a note in the doc on the dump-tables method might help send them down a good path. :-> Anyway, no biggie. :->

[jeromekelleher]

I agree with all the above. We might do the checks for a legal tree sequence before dumping, but add a flag to TableCollection.dump to disable this. That way it's possible to work with badly formed tables, but you have to go out of your way to do so.

[jeromekelleher]

I've had a look at this, and it's trickier than you might imagine given the way the lifetimes of tables within a table collection are handled in Python at the moment. I think this will have to be resolved first with proper read-only access to tables that are owned by a tree sequence.

[clank clank clank]

That was the sound of me kicking this can down the road to 0.6.2...

[jeromekelleher]

I think we can do this now and should pick it up.