Joss-reviews: [REVIEW]: SODAR Core

[whedon]

Submitting author: @mikkonie (Mikko Nieminen)
Repository: https://github.com/bihealth/sodar_core
Version: v0.8.4
Editor: @lpantano
Reviewer: @rbatorsky, @olgabot
Archive: 10.5281/zenodo.4269347

Status

Status badge code:

HTML: <a href="http://joss.theoj.org/papers/afd21b9f51bbce3fe2f0172d59d5fd33"><img src="http://joss.theoj.org/papers/afd21b9f51bbce3fe2f0172d59d5fd33/status.svg"></a>
Markdown: [![status](http://joss.theoj.org/papers/afd21b9f51bbce3fe2f0172d59d5fd33/status.svg)](http://joss.theoj.org/papers/afd21b9f51bbce3fe2f0172d59d5fd33)

Reviewers and authors:

Please avoid lengthy details of difficulties in the review thread. Instead, please create a new issue in the target repository and link to those issues (especially acceptance-blockers) by leaving comments in the review thread below. (For completists: if the target issue tracker is also on GitHub, linking the review thread in the issue or vice versa will create corresponding breadcrumb trails in the link target.)

Reviewer instructions & questions

@rbatorsky & @olgabot, please carry out your review in this issue by updating the checklist below. If you cannot edit the checklist please:

1. Make sure you're logged in to your GitHub account
2. Be sure to accept the invite at this URL: https://github.com/openjournals/joss-reviews/invitations

The reviewer guidelines are available here: https://joss.readthedocs.io/en/latest/reviewer_guidelines.html. Any questions/concerns please let @lpantano know.

✨ Please try and complete your review in the next two weeks ✨

Review checklist for @rbatorsky

Conflict of interest

• [x] As the reviewer I confirm that I have read the JOSS conflict of interest policy and that there are no conflicts of interest for me to review this work.

Code of Conduct

• [x] I confirm that I read and will adhere to the JOSS code of conduct.

General checks

• [x] Repository: Is the source code for this software available at the repository url?
• [x] License: Does the repository contain a plain-text LICENSE file with the contents of an OSI approved software license?
• [x] Version: v0.8.4
• [x] Authorship: Has the submitting author (@mikkonie) made major contributions to the software? Does the full list of paper authors seem appropriate and complete?

Functionality

• [x] Installation: Does installation proceed as outlined in the documentation?
• [x] Functionality: Have the functional claims of the software been confirmed?
• [x] Performance: If there are any performance claims of the software, have they been confirmed? (If there are no claims, please check off this item.)

Documentation

• [x] A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
• [x] Installation instructions: Is there a clearly-stated list of dependencies? Ideally these should be handled with an automated package management solution.
• [x] Example usage: Do the authors include examples of how to use the software (ideally to solve real-world analysis problems).
• [x] Functionality documentation: Is the core functionality of the software documented to a satisfactory level (e.g., API method documentation)?
• [x] Automated tests: Are there automated tests or manual steps described so that the function of the software can be verified?
• [x] Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support

Software paper

• [x] Authors: Does the paper.md file include a list of authors with their affiliations?
• [x] A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
• [x] References: Do all archival references that should have a DOI list one (e.g., papers, datasets, software)?

Review checklist for @olgabot

Conflict of interest

• [x] As the reviewer I confirm that I have read the JOSS conflict of interest policy and that there are no conflicts of interest for me to review this work.

Code of Conduct

• [x] I confirm that I read and will adhere to the JOSS code of conduct.

General checks

• [x] Repository: Is the source code for this software available at the repository url?
• [x] License: Does the repository contain a plain-text LICENSE file with the contents of an OSI approved software license?
• [x] Version: v0.8.4
• [x] Authorship: Has the submitting author (@mikkonie) made major contributions to the software? Does the full list of paper authors seem appropriate and complete?

Functionality

• [x] Installation: Does installation proceed as outlined in the documentation?
• [x] Functionality: Have the functional claims of the software been confirmed?
• [x] Performance: If there are any performance claims of the software, have they been confirmed? (If there are no claims, please check off this item.)

Documentation

• [x] A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
• [x] Installation instructions: Is there a clearly-stated list of dependencies? Ideally these should be handled with an automated package management solution.
• [x] Example usage: Do the authors include examples of how to use the software (ideally to solve real-world analysis problems).
• [x] Functionality documentation: Is the core functionality of the software documented to a satisfactory level (e.g., API method documentation)?
• [x] Automated tests: Are there automated tests or manual steps described so that the function of the software can be verified?
• [x] Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support

Software paper

• [x] Authors: Does the paper.md file include a list of authors with their affiliations?
• [x] A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
• [x] References: Do all archival references that should have a DOI list one (e.g., papers, datasets, software)?

[whedon]

Hello human, I'm @whedon, a robot that can help you with some common editorial tasks. @rbatorsky, @olgabot it looks like you're currently assigned to review this paper :tada:.

:star: Important :star:

If you haven't already, you should seriously consider unsubscribing from GitHub notifications for this (https://github.com/openjournals/joss-reviews) repository. As a reviewer, you're probably currently watching this repository which means for GitHub's default behaviour you will receive notifications (emails) for all reviews 😿

To fix this do the following two things:

1. Set yourself as 'Not watching' https://github.com/openjournals/joss-reviews:

1. You may also like to change your default settings for this watching repositories in your GitHub profile here: https://github.com/settings/notifications

For a list of things I can do to help you, just type:

@whedon commands

For example, to regenerate the paper pdf after making changes in the paper's md or bib files, type:

@whedon generate pdf

[whedon]

Attempting PDF compilation. Reticulating splines etc...

[whedon]

:point_right: Check article proof :page_facing_up: :point_left:

[olgabot]

Hello, it seems that I am not able to edit the first comment to complete the checklist. Could you add me as an editor for this PR?

@whedon commands

[danielskatz]

Please see the first comment in this issue - you need to accept the invitation...

[olgabot]

Ah sorry! I skipped that section somehow. Thank you!

[lpantano]

Hi @mikkonie, wanted to check if everything is ok, and you are not waiting for us but trying to address the issues opened by the reviewers.

Hi @olgabot, any chance you could update the checklist above for the items that are ok?

Thank you all!

[lpantano]

👋 @mikkonie, any update on this? if you are not able to work on this right now, we can pause this review until you can work again. Thanks.

[lpantano]

@mikkonie, I am going to pause this for now. Let me know when you have time to continue.

[mikkonie]

Hello,

Apologies for the radio silence @lpantano. I have actually made changes as suggested by your comments into the master branch of the project and commented on these changes in the issue tickets raised in our project. I missed the updates here and didn't realize I also had to respond in this issue.

Again, sorry about this. Please let me know if you have further comments regarding the issues as they've currently been handled.

[lpantano]

Thanks! @olgabot , @rbatorsky, let me know if you can continue with the review.

[rbatorsky]

Hi all,
Sorry, I missed the update.
I'll be able to review the changes and hopefully complete the review this week.
Thanks,
Rebecca

[olgabot]

Sorry, I missed the update as well. I'll take a look this week. Thank you for the ping!

[rbatorsky]

Hello,
I've gone through and read the updates. I submitted one issue about installation https://github.com/bihealth/sodar_core/issues/6 but other than that I think I've completed my review.
Best,
Rebecca

[lpantano]

Thank you @rbatorsky, @olgabot remember to check the boxes when you are finishing each point. Thanks!

[mikkonie]

Hello,

FYI, I made changes according to @rbatorsky's new issue and detailed them in the issue comments.

-Mikko

[lpantano]

Thank you. I will wait for @olgabot to finish the review. Thanks.

[olgabot]

Took a look at this, and have some questions/concerns:

• The release version is now 0.7.0, but this issue refers to 0.6.2 -- should this get updated in the issue?
• I've never used Django before, and I can't seem to get my mind around what this package does and how to use it, and thus cannot evaluate the functionality or performance. Continued the discussion here: https://github.com/bihealth/sodar_core/issues/4
• "Example usage: Do the authors include examples of how to use the software (ideally to solve real-world analysis problems)." (emphasis mine) -- It's not clear to me how this solves analysis problems. So far, I can tell that there is a web server but I don't understand how to set it up or how it can help solve analysis problems

[kthyng]

hi @mikkonie! Have you had a chance to address comments from @olgabot?

[danielskatz]

👋 @mikkonie - it looks like this review is waiting on you...

[mikkonie]

@olgabot and @danielskatz, I apologize for the delay in reacting to this.

Regarding the version number: there have indeed been some updates and I expect to tag version 0.7.1 before the holidays. If this does not cause issues from the review side, it sounds logical to update the version. Please let me know if something needs to be clarified regarding the new releases.

As for the usage of the package, I will try to elaborate on the context of the package more in the aforementioned discussion. Hopefully it helps.

On the real world analysis use case issue: there are publications in works which detail real life scientific use cases for this package. Sadly they have not been published yet, so we can't simply refer to them. One option we discussed internally is e.g. providing an example demo application which displays a scientific use case and how the package enables it. Would this sound helpful?

[lpantano]

I think is ok to update the version we can update it. I would say one case example should help here, from start to tend. @olgabot, please add anything else, if any. Thanks

[mikkonie]

As a heads up, I will be on vacation and mostly unavailable until 6.1. After that I'm happy to provide any more help and updates required to further the review. Also, in January it would be possible to use a published use case in the preprint stage in order to help illustrate use cases for this project.

[lpantano]

Hi @mikkonie, just wanted to check if you still need to work more on the paper before I ping the reviewers again. Just let us know when everything is ready, for a another round. Thanks! and happy new year!.

[holtgrewe]

@whedon generate pdf

[whedon]

:point_right: Check article proof :page_facing_up: :point_left:

[holtgrewe]

@rbatorsky @olgabot Thank you very much for your input. We think that we have addressed all of your points but would be glad to address specific issues if there are remaining ones.

@lpantano FYI we are ready to continue with the review and should be able to follow through with fewer delays after allocating more resources to this now.

[holtgrewe]

@whedon generate pdf

[whedon]

:point_right: Check article proof :page_facing_up: :point_left:

[lpantano]

Thanks for the update @holtgrewe, I'll give a couple of days to the reviewers to check whether they can continue with this now. Thanks!

[lpantano]

@rbatorsky, @olgabot, can you take a look at this and see whether the changes are good to continue.

[rbatorsky]

Thanks @lpantano. Sorry for the delay, I thought I had already given the final OK on Oct 6, 2019. Looks good to me and I have no further issues. Is there anything else I need to do?

[olgabot]

I'm still having trouble understanding what this tool does, and added a issue: https://github.com/bihealth/sodar_core/issues/11

[holtgrewe]

Hi I also wanted to report that it will take us 4 weeks to get back to this review because of another big revision in our group. We'll continue in the sodar_core ticket tracker with @olgabot

[olgabot]

Sounds good!

[arfon]

Dear authors and reviewers

We wanted to notify you that in light of the current COVID-19 pandemic, JOSS has decided to suspend submission of new manuscripts and to handle existing manuscripts (such as this one) on a "best efforts basis". We understand that you may need to attend to more pressing issues than completing a review or updating a repository in response to a review. If this is the case, a quick note indicating that you need to put a "pause" on your involvement with a review would be appreciated but is not required.

Thanks in advance for your understanding.

_Arfon Smith, Editor in Chief, on behalf of the JOSS editorial team._

[arfon]

:wave: just a friendly check-in to see how things are going with this review? @holtgrewe @mikkonie by the looks of things the reviewers are waiting on your input?

[arfon]

:wave: @holtgrewe @mikkonie - JOSS is now open again for submissions and we're actively trying to wrap up existing reviews. Do you think you might be able to address the reviewer feedback soon?

[kthyng]

Hi @holtgrewe and @mikkonie! I am going to pause this submission since it looks like the reviewers have done their work but we have been waiting on your attention for several months.

[arfon]

@whedon remind @arfon in three weeks

:wave: @holtgrewe @mikkonie - this submission has now not made any progress in more than six months. Unless we hear back on your intent to wrap this submission up in the next three weeks, we will move forward to withdraw the submission on your behalf.

[whedon]

@arfon doesn't seem to be a reviewer or author for this submission.

[holtgrewe]

@arfon thanks for the warning, it's actually on my calendar for today

[holtgrewe]

Dear all (in particular @olgabot and @arfon),

we're reallly sorry about the long delay but we got derailed by the usual and unusual things in academia...

Anyway, I think the main point to drive forward is:

• https://github.com/bihealth/sodar_core/issues/11

and our current planned change is

• https://github.com/bihealth/sodar_core/pull/14

There also is the raised point #12 but I'd actually limit the documentation here as described in the ticket documentation.

We are looking forward to completing the review process.

Best wishes,
Manuel

[holtgrewe]

Dear @arfon,
Dear @olgabot,

we are committed to wrapping this up ASAP. How can we ensure this?

Best wishes,
Manuel

[arfon]

:wave: @lpantano - I think this submission could do with your attention here.

[lpantano]

@holtgrewe, let me try to get the reviewer's attention again. Thanks.

[lpantano]

Hey @rbatorsky, authors closed the issue you had opened, I don't see any more concern from you, let us know if we are missing something. If all sounds good, could you make sure you double check all the items under the list on the top of this issue, please? Thanks!

[rbatorsky]

Thanks @lpantano! I've checked off all that apply. I don't believe that the submission described any automated tests. Is anything else needed from me?

[olgabot]

The most recent release is v0.8.3 -- should this be updated in the issue?

[lpantano]

@olgabot, yes, I will update that when all changes have been done and reviewers don't have more comments. thanks!

[olgabot]

Great, that was my final concern. The authors have addressed everything else. Excited to see this released!

[lpantano]

@rbatorsky, thank you! manual tests are ok as far as they work. In this case, probably it would be the example server. I think here, it would be this part: https://sodar-core.readthedocs.io/en/latest/for_the_impatient.html, if that tutorial works, I think we could check that box. Thank you!

[holtgrewe]

I'm happy to clarify that there are automated unit tests:

• https://travis-ci.org/github/bihealth/sodar_core/branches

The dev branch has some issues and probably needs an update of the chrome driver (sigh the sad story of automated web testing these days...). However the master branch builds nicely.

Cheers,
Manuel

[lpantano]

That really helps, thank you. @rbatorsky let us know.

[lpantano]

Hey @rbatorsky, could you quickly check that actually they have an automated test and it is working. Thanks!

https://travis-ci.org/github/bihealth/sodar_core/branches

[rbatorsky]

Thanks @lpantano and sorry for the delay. Yes, I can verify that the manual tutorial steps do work and the automated tests seem to be passing. I also checked the version number since the github issue 0.6.2 matches the paper, but as was mentioned there have been later releases. I think my items are all set.

[lpantano]

@whedon set v0.8.3 as version

[whedon]

OK. v0.8.3 is the version.

[lpantano]

@whedon generate pdf

[whedon]

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

[lpantano]

@whedon check references

[whedon]

Reference check summary (note 'MISSING' DOIs are suggestions that need verification):

OK DOIs

- 10.7287/peerj.preprints.27717v2 is OK
- 10.1101/2020.01.27.921965 is OK
- 10.1038/sdata.2016.18 is OK

MISSING DOIs

- None

INVALID DOIs

- None

[lpantano]

@holtgrewe, we are almost there, could you go over this list, please:

• [ ] Archive the reviewed software in Zenodo or a similar service (e.g., figshare, an institutional repository)
• [ ] Check the archival deposit (e.g., in Zenodo) has the correct metadata. This includes the title (should match the paper title) and author list (make sure the list is correct and people who only made a small fix are not on it). You may also add the authors' ORCID.
• [ ] Please list the DOI of the archived version here.

After that, I can then move forward with accepting the submission :)

[holtgrewe]

Dear @lpantano, I'm happy to report that we made a v0.8.4 release of the software and deposited it here in Zenodo:

• https://zenodo.org/record/4269347#.X6z2564o-Ak

The DOI is 10.5281/zenodo.4269347 and we already updated the paper.md file in the sodar core repository to reflect that version update as well as the codemeta.json file.

Please tell us if we missed anything ;-)

[lpantano]

@whedon set v0.8.4 as version

[whedon]

OK. v0.8.4 is the version.

[lpantano]

@whedon set 10.5281/zenodo.4269347 as archive

[whedon]

OK. 10.5281/zenodo.4269347 is the archive.

[lpantano]

@whedon accept

[whedon]

Attempting dry run of processing paper acceptance...

[whedon]

Reference check summary (note 'MISSING' DOIs are suggestions that need verification):

OK DOIs

- 10.1093/bioinformatics/btz850 is OK
- 10.1093/nar/gkaa241 is OK
- 10.1038/sdata.2016.18 is OK

MISSING DOIs

- None

INVALID DOIs

- None

[whedon]

:wave: @openjournals/joss-eics, this paper is ready to be accepted and published.

Check final proof :point_right: https://github.com/openjournals/joss-papers/pull/1906

If the paper PDF and Crossref deposit XML look good in https://github.com/openjournals/joss-papers/pull/1906, then you can now move forward with accepting the submission by compiling again with the flag deposit=true e.g.
@whedon accept deposit=true

[lpantano]

Thank you everybody for the hard work, let's see what the editorial board says. Really appreciated the work from everybody.

[kthyng]

@whedon accept deposit=true

[whedon]

Doing it live! Attempting automated processing of paper acceptance...

[whedon]

🐦🐦🐦 👉 Tweet for this paper 👈 🐦🐦🐦

[whedon]

🚨🚨🚨 THIS IS NOT A DRILL, YOU HAVE JUST ACCEPTED A PAPER INTO JOSS! 🚨🚨🚨

Here's what you must now do:

1. Check final PDF and Crossref metadata that was deposited :point_right: https://github.com/openjournals/joss-papers/pull/1913
2. Wait a couple of minutes to verify that the paper DOI resolves https://doi.org/10.21105/joss.01584
3. If everything looks good, then close this review issue.

4. Party like you just published a paper! 🎉🌈🦄💃👻🤘

   Any issues? Notify your editorial technical team...

[kthyng]

Congrats on your new publication @mikkonie! Thanks to editor @lpantano and reviewers @rbatorsky and @olgabot for your time and expertise!! 🎉 🎉

[whedon]

:tada::tada::tada: Congratulations on your paper acceptance! :tada::tada::tada:

If you would like to include a link to your paper from your README use the following code snippets:

Markdown:
[![DOI](https://joss.theoj.org/papers/10.21105/joss.01584/status.svg)](https://doi.org/10.21105/joss.01584)

HTML:
<a style="border-width:0" href="https://doi.org/10.21105/joss.01584">
  <img src="https://joss.theoj.org/papers/10.21105/joss.01584/status.svg" alt="DOI badge" >
</a>

reStructuredText:
.. image:: https://joss.theoj.org/papers/10.21105/joss.01584/status.svg
   :target: https://doi.org/10.21105/joss.01584

This is how it will look in your documentation:

We need your help!

Journal of Open Source Software is a community-run journal and relies upon volunteer effort. If you'd like to support us please consider doing either one (or both) of the the following:

• Volunteering to review for us sometime in the future. You can add your name to the reviewer list here: https://joss.theoj.org/reviewer-signup.html
• Making a small donation to support our running costs here: https://numfocus.org/donate-to-joss