Stdpopsim: use sim_mutations and sim_ancestry

[mufernando]

msprime deprecated msprime.simulate so we need to update both the slim engine and msprime engine to use the new sim_ancestry and sim_mutation functions.

[jeromekelleher]

Do we have a dev branch for msprime 1.0 changes?

[grahamgower]

Yes, there's an msprime-1.0 branch with some code on it that requires 1.0 features. But msprime moved on a bit, and now stdpopsim doesn't work with the msprime 1.0 alpha release because pyslim needs to be updated. :) Fun times.

[jeromekelleher]

Crud.

[petrelharp]

I'm working on a release, but have noticed problems with my 'fix' for this problem: https://github.com/tskit-dev/pyslim/pull/144 - ... but I am not sure you actually need me to do anything, since I don't think we should be using recapitate with msprime 1.0, at least for now?

Recall that the problem was that we wanted to be able to use under-development discrete recombination maps with recapitate, and so had a hack in to check for these and if so, use them. So, I wondered if I should modify recapitate so that it would check for the presence of msprime.sim_ancestry, and if so, use it? Since we'll want to use sim_ancestry instead of simulate here, as in the title of this issue. In other words, pass **kwargs on to sim_ancestry instead of simulate, if possible.

However, looking carefully at recapitate, much of what it formerly needed to do has been made obsolete by tskit metadata (yay), and the only important thing it's doing is maintaining the reference sequence.

So: I can make this error not occur any more, but that doesn't let you move forward here because you actually need to use sim_ancestry, which recapitate doesn't currently let you do. And, figuring out how to hook recapitate up to sim_ancestry is a bit tricky, as for instance, suppose someone passes in a mutation rate; should I then also call sim_mutations? (I think the answer is "gah that's terrible", so maybe recapitate will just have to be not-entirely-backwards-compatible, and so will have to wait for msprime 1.0, and not be available as a release beforehand.)

tldr; I think we don't need recapitate here, and should just do the underlying calls to msprime directly, making sure to bring slim_generation and reference_sequence along.

[jeromekelleher]

tldr; I think we don't need recapitate here, and should just do the underlying calls to msprime directly, making sure to bring slim_generation and reference_sequence along.

+1 - that sounds like the most straightforward appoach!

Re pyslim, we should still be able to call msprime.simulate(). If we pass in a recombination map with num_loci==L it should all work as before.

Put it this way: 1.0 is aiming for 100% compatibility with 0.7.5. If it works now, it should continue to work with 1.0.

[mufernando]

we haven't officially released the SLiM engine in stdpopsim, right? so I guess we can do what you suggested in the tldr and use msprime sim_ancestry and _mutations directly.

[grahamgower]

The SLiM engine is in the stable 0.1.2 of stdpopsim. So we'll want to make sure that doesn't get broken by any new pyslim/msprime releases. But, absolutely, we can make stdpopsim do discrete recombination maps using the new msprime API directly.

[grahamgower]

Current situation with the stable stdpopsim release and msprime 1.0.0a4:

$ stdpopsim -e slim --slim-scaling-factor 10 --slim-burn-in 0.1 HomSap -c chr22 10 > /tmp/foo.ts
Simulation information:
    Engine: slim (3.5)
    Model id: PiecewiseConstant
    Model desciption: Piecewise constant size population model over multiple epochs.
    Population: number_samples (sampling_time_generations):
        pop0: 10 (0)
Contig Description:
    Contig length: 51304566.0
    Mean recombination rate: 1.4445022767788227e-08
    Mean mutation rate: 1.29e-08
    Genetic map: None

WARNING: You're using a scaling factor (10.0). This should give similar results for many situations, but is not equivalent, especially in the presence of selection. When using rescaling, you should be careful---do checks and compare results across different values of the scaling factor.
Traceback (most recent call last):
  File "/home/grg/.local/bin/stdpopsim", line 8, in <module>
    sys.exit(stdpopsim_main())
  File "/home/grg/.local/lib/python3.9/site-packages/stdpopsim/cli.py", line 651, in stdpopsim_main
    run(args)
  File "/home/grg/.local/lib/python3.9/site-packages/stdpopsim/cli.py", line 642, in run
    args.runner(args)
  File "/home/grg/.local/lib/python3.9/site-packages/stdpopsim/cli.py", line 457, in run_simulation
    ts = engine.simulate(**kwargs)
  File "/home/grg/.local/lib/python3.9/site-packages/stdpopsim/slim_engine.py", line 725, in simulate
    ts = self._recap_and_rescale(
  File "/home/grg/.local/lib/python3.9/site-packages/stdpopsim/slim_engine.py", line 804, in _recap_and_rescale
    ts = ts.recapitate(
  File "/home/grg/.local/lib/python3.9/site-packages/pyslim/slim_tree_sequence.py", line 473, in recapitate
    recap = msprime.simulate(
TypeError: simulate() got an unexpected keyword argument 'discrete_genome'

$ pip show stdpopsim msprime pyslim
Name: stdpopsim
Version: 0.1.2
Summary: A library of population genetic simulation models
Home-page: https://github.com/popsim-consortium/stdpopsim
Author: PopSim Consortium
Author-email: [email protected]
License: GNU GPLv3+
Location: /home/grg/.local/lib/python3.9/site-packages
Requires: attrs, msprime, appdirs, humanize, pyslim
Required-by:
---
Name: msprime
Version: 1.0.0a4
Summary: A fast and accurate coalescent simulator.
Home-page: https://pypi.org/project/msprime/
Author: Tskit Developers
Author-email: [email protected]
License: GNU GPLv3+
Location: /home/grg/.local/lib/python3.9/site-packages
Requires: tskit, newick, numpy
Required-by: stdpopsim, pyslim
---
Name: pyslim
Version: 0.501
Summary: Manipulate tree sequences produced by SLiM.
Home-page: https://github.com/tskit-dev/pyslim
Author: Peter Ralph
Author-email: [email protected]
License: MIT
Location: /home/grg/.local/lib/python3.9/site-packages
Requires: tskit, numpy, msprime, attrs, kastore
Required-by: stdpopsim

[EDIT: but it works fine with dev pyslim as of this morning, installed via pip install git+https://github.com/tskit-dev/pyslim.git#egg=pyslim. Woohoo!]

[jeromekelleher]

Closed in recent updates.