As discussed over in #924, keeping the full tree sequence that's provided as input to msprime
with from_ts is probably not a great idea.
We can probably do something equivalent that's more elegant though by storing the offsets of the nodes and edges of the input tables. The original tree sequence should then be recoverable.
If from_ts's provenance is stored in provenances[i-1] (as it is now) then from_ts should be recoverable from that? Is that good enough?
I think that 'simulate from' should add one additional entry to the provenance table, like other operations. That's not what you're suggesting, is it?
I'll come up with a proposal this morning and report back - easiest way to see if what I have in mind will work.
It'd be nice to get this fixed up soon, because people using the SLiM engine in stdpopsim are going to end up with giant tree sequences. For instance, in this fairly small example, it looks like the provenance is ~90% of the file size (if the numbers are accurate?):
import stdpopsim
species = stdpopsim.get_species("HomSap")
contig = species.get_contig("chr22", length_multiplier=0.1)
# default is a flat genetic map
model = species.get_demographic_model("Africa_1T12")
samples = model.get_samples(200)
engine = stdpopsim.get_engine("slim")
ts = engine.simulate(model, contig, samples, slim_scaling_factor=10)
# The provenance information for the resulting tree sequence is 44.22MB. This is nothing to worry about as provenance is a good thing to have, but if you want to save this memory/storage space you can disable provenance recording by setting record_provenance=False
ts.dump("test.trees")
and then
$ ls -lth test.trees
-rw-r--r-- 1 peter peter 48M Apr 2 20:49 test.trees
Fair enough. @benjeffery, I guess the simplest thing to do for now is just remove the ts from the provenance, and skip any round-tripping tests? We can follow up on the right way to say "current_ts" at some other point.
@jeromekelleher Yep, makes sense - will do a PR later.
Most helpful comment
@jeromekelleher Yep, makes sense - will do a PR later.