Joss-reviews: [REVIEW]: Virtual Scanner: MRI on a Browser

[whedon]

Submitting author: @imr-framework (Sairam Geethanath)
Repository: https://github.com/imr-framework/virtual-scanner
Version: 1.0.0
Editor: @arokem
Reviewers: @nstikov, @vsoch, @mathieuboudreau
Archive: 10.5281/zenodo.3550315

Status

Status badge code:

HTML: <a href="https://joss.theoj.org/papers/45280f505cac55a966f8842a651ef48a"><img src="https://joss.theoj.org/papers/45280f505cac55a966f8842a651ef48a/status.svg"></a>
Markdown: [![status](https://joss.theoj.org/papers/45280f505cac55a966f8842a651ef48a/status.svg)](https://joss.theoj.org/papers/45280f505cac55a966f8842a651ef48a)

Reviewers and authors:

Please avoid lengthy details of difficulties in the review thread. Instead, please create a new issue in the target repository and link to those issues (especially acceptance-blockers) by leaving comments in the review thread below. (For completists: if the target issue tracker is also on GitHub, linking the review thread in the issue or vice versa will create corresponding breadcrumb trails in the link target.)

Reviewer instructions & questions

@nstikov, @mathieuboudreau, & @vsoch, please carry out your review in this issue by updating the checklist below. If you cannot edit the checklist please:

1. Make sure you're logged in to your GitHub account
2. Be sure to accept the invite at this URL: https://github.com/openjournals/joss-reviews/invitations

The reviewer guidelines are available here: https://joss.readthedocs.io/en/latest/reviewer_guidelines.html. Any questions/concerns please let @arokem know.

✨ Please try and complete your review in the next two weeks ✨

Review checklist for @nstikov & @mathieuboudreau

Conflict of interest

• [x] As the reviewer I confirm that I have read the JOSS conflict of interest policy and that there are no conflicts of interest for me to review this work.

Code of Conduct

• [x] I confirm that I read and will adhere to the JOSS code of conduct.

General checks

• [x] Repository: Is the source code for this software available at the repository url?
• [x] License: Does the repository contain a plain-text LICENSE file with the contents of an OSI approved software license?
• [x] Version: Does the release version given match the GitHub release (1.0.0)?
• [x] Authorship: Has the submitting author (@imr-framework) made major contributions to the software? Does the full list of paper authors seem appropriate and complete?

Functionality

• [x] Installation: Does installation proceed as outlined in the documentation?
• [x] Functionality: Have the functional claims of the software been confirmed?
• [x] Performance: If there are any performance claims of the software, have they been confirmed? (If there are no claims, please check off this item.)

Documentation

• [x] A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
• [x] Installation instructions: Is there a clearly-stated list of dependencies? Ideally these should be handled with an automated package management solution.
• [x] Example usage: Do the authors include examples of how to use the software (ideally to solve real-world analysis problems).
• [x] Functionality documentation: Is the core functionality of the software documented to a satisfactory level (e.g., API method documentation)?
• [x] Automated tests: Are there automated tests or manual steps described so that the function of the software can be verified?
• [x] Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support

Software paper

• [x] Authors: Does the paper.md file include a list of authors with their affiliations?
• [x] A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
• [x] References: Do all archival references that should have a DOI list one (e.g., papers, datasets, software)?

Review checklist for @vsoch

Conflict of interest

• [x] As the reviewer I confirm that I have read the JOSS conflict of interest policy and that there are no conflicts of interest for me to review this work.

Code of Conduct

• [x] I confirm that I read and will adhere to the JOSS code of conduct.

General checks

• [x] Repository: Is the source code for this software available at the repository url?
• [x] License: Does the repository contain a plain-text LICENSE file with the contents of an OSI approved software license?
• [x] Version: Does the release version given match the GitHub release (1.0.0)?
• [x] Authorship: Has the submitting author (@imr-framework) made major contributions to the software? Does the full list of paper authors seem appropriate and complete?

Functionality

• [x] Installation: Does installation proceed as outlined in the documentation?
• [x] Functionality: Have the functional claims of the software been confirmed?
• [x] Performance: If there are any performance claims of the software, have they been confirmed? (If there are no claims, please check off this item.)

Documentation

• [x] A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
• [x] Installation instructions: Is there a clearly-stated list of dependencies? Ideally these should be handled with an automated package management solution.
• [x] Example usage: Do the authors include examples of how to use the software (ideally to solve real-world analysis problems).
• [x] Functionality documentation: Is the core functionality of the software documented to a satisfactory level (e.g., API method documentation)?
• [ ] Automated tests: Are there automated tests or manual steps described so that the function of the software can be verified?
• [x] Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support

Software paper

• [x] Authors: Does the paper.md file include a list of authors with their affiliations?
• [x] A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
• [x] References: Do all archival references that should have a DOI list one (e.g., papers, datasets, software)?

[whedon]

Hello human, I'm @whedon, a robot that can help you with some common editorial tasks. @nstikov, @vsoch it looks like you're currently assigned to review this paper :tada:.

:star: Important :star:

If you haven't already, you should seriously consider unsubscribing from GitHub notifications for this (https://github.com/openjournals/joss-reviews) repository. As a reviewer, you're probably currently watching this repository which means for GitHub's default behaviour you will receive notifications (emails) for all reviews 😿

To fix this do the following two things:

1. Set yourself as 'Not watching' https://github.com/openjournals/joss-reviews:

1. You may also like to change your default settings for this watching repositories in your GitHub profile here: https://github.com/settings/notifications

For a list of things I can do to help you, just type:

@whedon commands

For example, to regenerate the paper pdf after making changes in the paper's md or bib files, type:

@whedon generate pdf

[whedon]

Attempting PDF compilation. Reticulating splines etc...

[whedon]

:point_right: Check article proof :page_facing_up: :point_left:

[vsoch]

Wowza this is my third this week, will get started when I can!

[mathieuboudreau]

Just as a clarifying point, I am reviewing this in tandem with @nstikov (I'm a research fellow of his), and will likely be the one most active out of both of us in this discussion/review on GitHub. This was determined in the pre-review (#1590)

[vsoch]

Great, thanks for the update @mathieuboudreau.

[mathieuboudreau]

@vsoch is it possible to make me able to edit the checklist status?

[vsoch]

You can't click in the top right and see this menu?

If not, I think @arokem would need to add you as a collaborator or with appropriate permissions to do that. Worst case, I can offer to update it for you (note that I'm another reviewer, not an editor).

[mathieuboudreau]

Ahh sorry, I didn't realize your were another reviewer, sorry about that!

No, I don't have access to Edit, as only @nstikov was assigned as the sole reviewer instead of both of us for "his" review.

@arokem can this be resolved?

[arokem]

@whedon add @mathieuboudreau as reviewer

[whedon]

OK, @mathieuboudreau is now a reviewer

[arokem]

Sorry about that. Does it work now that I added you as a reviewer?

[mathieuboudreau]

I just checked, and no unfortunately not.

[mathieuboudreau]

If it's too much trouble, I could just keep track of them and ask @nstikov to check them off in bulk periodically.

[arokem]

How about now, that I've added a section for you at the top.

[mathieuboudreau]

Still unable, sorry.

[vsoch]

@mathieuboudreau here is the markdown https://gist.github.com/vsoch/c9f65143293acb975af251608b314579

Just view the raw, make changes somewhere that I can view raw (another gist) and I'll copy paste it in for you.

[danielskatz]

@arfon needs to invite @mathieuboudreau to the org...

[arfon]

@mathieuboudreau - did you accept in the invite to the organization here? https://github.com/openjournals/joss-reviews/invitations

[mathieuboudreau]

Ahh, I must of missed the notification of that invite - my apologies. It works now!

[tonggehua]

@whedon generate pdf

[whedon]

Attempting PDF compilation. Reticulating splines etc...

[whedon]

:point_right: Check article proof :page_facing_up: :point_left:

[arokem]

Where do things stand here?

@vsoch: are you able to check any of the boxes on your list?

@mathieuboudreau and @nstikov: from your end, I see that the only remaining points are installation and version. Not worrying about version for the moment, what still needs to be done for the installation check box? Is there an associated issue on the project repo?

[vsoch]

I'm still reviewing - I'll check them when I'm finished with the issues above! For example, I'm waiting to hear back about testing (issue 38 linked above). I definitely am not dragging feet or trying to be slow, I'm bringing up issues and the authors are addressing them, and I'll update the final review here when that is done.

[mathieuboudreau]

There is an issue on my end, but the authors closed it without it fully being resolved. It's https://github.com/imr-framework/virtual-scanner/issues/29

Their "Quick Start" installation instructions are still broken as of the last time I tried it out.

[arokem]

Thanks for the feedback!

Their "Quick Start" installation instructions are still broken as of the last time I tried it out.

@imr-framework : I agree with @mathieuboudreau. Looks like that issue is still not resolved (despite an impressive amount of work on the part of the reviewer and the authors to resolve this!). Could you please work to resolve this issue? It looks like it's getting closer...

[mathieuboudreau]

If it does get too cumbersome to fix this issue, I'm fine with them just omitting that section from their README, as there are two other methods of installations described on there that do work. I just don't want there to be instructions that don't work listed (particular the "easiest" one, which might discourage new users).

[mathieuboudreau]

And yes, I very much appreciate the many efforts the authors have invested in resolving it!

[vsoch]

+1! And this is a really good (real life example) for why I pushed hard for containers. Virtual environments are better than nothing, but they aren't reproducible in the same way a container is.

[arokem]

I would be fine with a change just to the docs, if you can't address all of these issues.

[arokem]

@imr-framework : have you had a chance to address this?

[mathieuboudreau]

@imr-framework has resolved the outstanding issues I had with the installation instructions, and I've now completed all of the items on my checklist. Thanks a lot @imr-framework for making my suggested fixes/desired features along the way!

[arokem]

Thanks @mathieuboudreau ! @vsoch : I see that none of your boxes are checked at this point. Have you had a chance to take a look? Do you still have any concerns or unadressed issues?

[vsoch]

oups, sorry about that! I think I must have checked off the boxes in a browser tab many weeks ago and forgot to press save :)

I've updated my checkboxes, the only missing item is automated tests, represented in this issue https://github.com/imr-framework/virtual-scanner/issues/38

[arokem]

Awesome. Thanks!

@imr-framework: could you please address the remaining issue?

[arokem]

@imr-framework : have you had a chance to address this issue?

[tonggehua]

@arokem Sorry for the late reply. My research group is focusing on a conference abstract deadline (Nov. 6) and we plan to address the automated testing issue immediately after it. Most back-end tests are done, but setting up proper front-end testing (with Selenium) will take more time.

[labarba]

@imr-framework — Could we have a status update of your work on responding to reviewer comments? Let me know if you want us to set a reminder for you.

[tonggehua]

@labarba We have got the front end testing working on Windows. Currently, we are testing it on mac and wrapping it up in a test case for Github actions to run it along with all the back end tests. The goal is to have this done by Friday.

[vsoch]

@labarba the authors have tests that work locally but not on CI (not automated), and have requested that the requirement is waived. This is below my personal standard and review criteria, but I do recognize they put some effort into trying. I won't check the automated tests box or close this issue, but I'll forward the final decision on to you.

[sairamgeethanath]

@labarba Pl. also refer our response: https://github.com/imr-framework/virtual-scanner/issues/38#issuecomment-556128982

[arokem]

I think that based on our review criteria this falls in the "OK" category.

[arokem]

@whedon check references

[whedon]

Attempting to check references...

[arokem]

@whedon generate pdf

[whedon]

Attempting PDF compilation. Reticulating splines etc...

[whedon]

:point_right: Check article proof :page_facing_up: :point_left:

[whedon]

```Reference check summary:

OK DOIs

• 10.1002/mrm.26235 is OK
• 10.1016/j.mri.2018.03.008 is OK
• 10.1016/j.jmr.2017.05.007 is OK
• 10.1002/mrm.24138 is OK
• 10.1002/mrm.1910150117 is OK
• 10.1126/science.1925560 is OK
• 10.1002/jmri.26638 is OK
• 10.1002/mrm.22406 is OK
• 10.1002/mrm.25528 is OK
• 10.1088/0022-3719/10/3/004 is OK
• 10.1007/s12045-015-0268-2 is OK

MISSING DOIs

• None

INVALID DOIs

• None
  ```

[arokem]

@openjournals/joss-eics : I believe this paper is ready for your review for acceptance.

[kthyng]

Hi @imr-framework! Please archive your work in something like Zenodo and report the doi back here.

[kthyng]

Another note: Your paper looks good, but the capitalization in your references is off. When you want to preserve capitalization in bibtex references, you can add {} around the word or phrase. For example, I see "nmr" several times and I believe this should be capitalized. You can preserve that with {NMR} in the bibtex entry. Could you go through your references and make sure they come through to the paper looking correct in the references list?

[tonggehua]

@whedon generate pdf

[whedon]

Attempting PDF compilation. Reticulating splines etc...

[whedon]

:point_right: Check article proof :page_facing_up: :point_left:

[tonggehua]

@whedon generate pdf

[whedon]

Attempting PDF compilation. Reticulating splines etc...

[whedon]

:point_right: Check article proof :page_facing_up: :point_left:

[kthyng]

great, @tonggehua, this looks better. Report it back here when you have your Zenodo archive.

[tonggehua]

@whedon generate pdf

[whedon]

Attempting PDF compilation. Reticulating splines etc...

[whedon]

:point_right: Check article proof :page_facing_up: :point_left:

[tonggehua]

@kthyng Here is the DOI : 10.5281/zenodo.3550315

[kthyng]

@whedon set 10.5281/zenodo.3550315 as archive

[whedon]

OK. 10.5281/zenodo.3550315 is the archive.

[kthyng]

@whedon accept

[whedon]

Attempting dry run of processing paper acceptance...

[whedon]

```Reference check summary:

OK DOIs

• 10.1002/mrm.26235 is OK
• 10.1016/j.mri.2018.03.008 is OK
• 10.1016/j.jmr.2017.05.007 is OK
• 10.1002/mrm.24138 is OK
• 10.1002/mrm.1910150117 is OK
• 10.1126/science.1925560 is OK
• 10.1002/jmri.26638 is OK
• 10.1002/mrm.22406 is OK
• 10.1002/mrm.25528 is OK
• 10.1088/0022-3719/10/3/004 is OK
• 10.1007/s12045-015-0268-2 is OK

MISSING DOIs

• https://doi.org/10.1016/j.crad.2003.09.015 may be missing for title: MRI from Picture to Proton

INVALID DOIs

• None
  ```

[whedon]

Check final proof :point_right: https://github.com/openjournals/joss-papers/pull/1129

If the paper PDF and Crossref deposit XML look good in https://github.com/openjournals/joss-papers/pull/1129, then you can now move forward with accepting the submission by compiling again with the flag deposit=true e.g.
@whedon accept deposit=true

[kthyng]

@imr-framework, @tonggehua I have added the doi stated as missing in the previous references check in PR 43 if you want to merge

[tonggehua]

@kthyng Thank you! I merged it.

[kthyng]

@whedon generate pdf

[whedon]

Attempting PDF compilation. Reticulating splines etc...

[whedon]

:point_right: Check article proof :page_facing_up: :point_left:

[kthyng]

@whedon accept

[whedon]

Attempting dry run of processing paper acceptance...

[whedon]

```Reference check summary:

OK DOIs

• 10.1002/mrm.26235 is OK
• 10.1016/j.mri.2018.03.008 is OK
• 10.1016/j.jmr.2017.05.007 is OK
• 10.1002/mrm.24138 is OK
• 10.1002/mrm.1910150117 is OK
• 10.1126/science.1925560 is OK
• 10.1002/jmri.26638 is OK
• 10.1002/mrm.22406 is OK
• 10.1002/mrm.25528 is OK
• 10.1088/0022-3719/10/3/004 is OK
• 10.1007/s12045-015-0268-2 is OK
• 10.1016/j.crad.2003.09.015 is OK

MISSING DOIs

• None

INVALID DOIs

• None
  ```

[whedon]

Check final proof :point_right: https://github.com/openjournals/joss-papers/pull/1130

If the paper PDF and Crossref deposit XML look good in https://github.com/openjournals/joss-papers/pull/1130, then you can now move forward with accepting the submission by compiling again with the flag deposit=true e.g.
@whedon accept deposit=true

[kthyng]

@whedon accept deposit=true

[whedon]

Doing it live! Attempting automated processing of paper acceptance...

[whedon]

🐦🐦🐦 👉 Tweet for this paper 👈 🐦🐦🐦

[whedon]

🚨🚨🚨 THIS IS NOT A DRILL, YOU HAVE JUST ACCEPTED A PAPER INTO JOSS! 🚨🚨🚨

Here's what you must now do:

1. Check final PDF and Crossref metadata that was deposited :point_right: https://github.com/openjournals/joss-papers/pull/1131
2. Wait a couple of minutes to verify that the paper DOI resolves https://doi.org/10.21105/joss.01637
3. If everything looks good, then close this review issue.

4. Party like you just published a paper! 🎉🌈🦄💃👻🤘

   Any issues? notify your editorial technical team...

[kthyng]

Congratulations to @imr-framework, @tonggehua on your new paper!!

Thanks to @arokem for editing and @nstikov, @vsoch, @mathieuboudreau for reviewing... this wouldn't be possible without your time and expertise! 🎉 🎉

[whedon]

:tada::tada::tada: Congratulations on your paper acceptance! :tada::tada::tada:

If you would like to include a link to your paper from your README use the following code snippets:

Markdown:
[![DOI](https://joss.theoj.org/papers/10.21105/joss.01637/status.svg)](https://doi.org/10.21105/joss.01637)

HTML:
<a style="border-width:0" href="https://doi.org/10.21105/joss.01637">
  <img src="https://joss.theoj.org/papers/10.21105/joss.01637/status.svg" alt="DOI badge" >
</a>

reStructuredText:
.. image:: https://joss.theoj.org/papers/10.21105/joss.01637/status.svg
   :target: https://doi.org/10.21105/joss.01637

This is how it will look in your documentation:

We need your help!

Journal of Open Source Software is a community-run journal and relies upon volunteer effort. If you'd like to support us please consider doing either one (or both) of the the following:

• Volunteering to review for us sometime in the future. You can add your name to the reviewer list here: https://joss.theoj.org/reviewer-signup.html
• Making a small donation to support our running costs here: https://numfocus.org/donate-to-joss

[vsoch]

Congrats @imr-framework, @tonggehua you guys worked hard on this!

[mathieuboudreau]

Congrats - well done!