Openff-toolkit: Best practice for reproducible saving of parameterized drug

[ljmartin]

Hi all,
With the amazing rapid development and new versions I thought it best to check how you implement reproducible drug parameterization for research and publication, which might span 6-12 months and potentially a few changes in openforcefield from start to end. Obviously this is an issue with any software package but OFF seems to be moving quickly.

My current approach: create an OpenMM system using ff.create_openmm_system(off_topology), then saving using the openmm XmlSerializer. This looks like it has all the relevant parameters within a single file and OpenMM doesn't look likely to change how they use these files/objects.
that is:

with open('drug_system.xml', 'w') as f:
    f.write(
           XmlSerializer.serialize( drug_system )
    )

However, there's also the openmm Forcefield XML file and the openforcefield .offxml files.

Any advice on what is best practice here would be helpful, Thanks!
lewis

[j-wags]

I'm writing docs this week, and this is a good reminder to include guidance on reproducibility in there as well!

The serialized XML approach that you show above is one way to record the parameters that you use. However, I think we're going to officially advocate for a more descriptive "reproducibility checklist" in the medium term.

Generally, the checklist for reproducible parameterization is:

• [ ] A record of the OFFXML file used -- If this is unchanged from the download, the filename will suffice. We make a big deal about never changing the contents of an "official" FF after it is released, and static backup of each one is stored beyond our reach on Zenodo (each of our "official FFs" has a Zenodo DOI)
• [ ] The original molecule file(s) that make up the system
• [ ] The Python script used to perform the parameterization
• [ ] The output of conda list (which will encapsulate the OFF Toolkit version, as well as dependencies)

In most circumstances, these four items should be a complete reproducible record of the parameterization that was performed. Moving forward, we're thinking about streamlining all of these into a single "provenance package" that is automatically produced during runs, and could be submitted to a journal as SI.

Please let me know if this addresses your question. Also, pinging @davidlmobley @SimonBoothroyd and @jchodera to see if they agree with the checklist above

[davidlmobley]

Yes, I agree with this!

[jchodera]

My current approach: create an OpenMM system using ff.create_openmm_system(off_topology), then saving using the openmm XmlSerializer. This looks like it has all the relevant parameters within a single file and OpenMM doesn't look likely to change how they use these files/objects.

This is also a good strategy to ensure you can stably reproduce the same simulation (to within statistical error), though it is harder for someone to pick up your work to modify something in a useful way. I'd suggest both strategies (yours and what @j-wags suggests above) if possible!

You can also gzip the OpenMM .xml files to save space:

import gzip
with gzip.open('drug_system.xml', 'w') as f:
    f.write(
           XmlSerializer.serialize( drug_system )
    )

[ljmartin]

Yes that answers it - thankyou!

[jaimergp]

@j-wags: The output of conda list (which will encapsulate the OFF Toolkit version, as well as dependencies)

I'd suggest using the output of conda env export instead, which has the same information, but consumable in a programmatic way with conda env create -f env.yaml!

[j-wags]

Ahhh, right -- conda env export is a much better idea.

[jaimergp]

Oh, and for the sake of reproducibility, conda info wouldn't hurt either.