Joss: JOSS for replication studies?

[braverock]

JOSS makes a great place for packages or libraries of open source software.

I'm wondering whether JOSS is also a good place for replication of prior work. Many published papers do not publish their code, so future researchers must reimplement the code from often-scanty hints, equations, and if-lucky pseudo-code from the original paper.

It seems that JOSS could be a good home for the software replication of other published studies where code was not originally provided.

@arfon , thoughts?

[arokem]

I'm not sure this is within scope. This sounds a lot like what Rescience is doing: http://rescience.github.io/

[labarba]

I second what @arokem says, but add that it's certainly possible to submit to JOSS the _software_ written alongside a replication study. That means a different sort of peer review on the software product, via the JOSS process.

[braverock]

@labarba , that's what I was getting at. The software created for a replication study may not be sufficient or stand-alone enough to warrant a separately released library or package.

Similarly, a researcher may wish to replicate the method from another paper as part of a larger work. That larger work may eventually get published, certainly, but I think the replication of the previously published method in software itself has merit.

Whether the ReScience project is a better venue for replication studies that include software (which all of theirs seem like they should) is a slightly different question.

[labarba]

The _replication study_ would not be in scope to be published in JOSS. It can be submitted to ReScience.

The _software_, to be published in JOSS, should satisfy the JOSS requirements of being feature complete, having a research goal, being well documented and all the other features JOSS reviewers will look for.

[arokem]

This returns to issues brought up way back in https://github.com/openjournals/joss/issues/52

For example, here is a Rescience paper my colleagues and I wrote:

https://zenodo.org/record/495237

This paper used software that was implemented in a pull request on a much larger software project:

https://github.com/nipy/dipy/pull/835

It seems to hit all of the requirements that @labarba mentioned, but it's part of a larger software project. What would be the workflow to publish a JOSS paper based specifically on this PR/feature? Does our current workflow already support that?

[labarba]

@arokem : Your question appears to be different than the original issue. If you feel you'd like to open a new issue for it, please go ahead. But for now, I'm going to close this issue, as the original question had been answered. (If I'm mistaken, feel free to re-open.)

[arokem]

Thanks! Looks like #52 is still open, so we can continue that discussion there.