Joss-reviews: [REVIEW]: ENZO: An Adaptive Mesh Refinement Code for Astrophysics (Version 2.6)

Created on 9 Aug 2019 · 74Comments · Source: openjournals/joss-reviews

Submitting author: @bwoshea (Brian O'Shea)
Repository: https://github.com/enzo-project/enzo-dev
Version: v2.6.1
Editor: @danielskatz
Reviewer: @zingale, @rtfisher
Archive: 10.5281/zenodo.3469922

Status

Status badge code:

HTML: <a href="https://joss.theoj.org/papers/ee73bb825d396e5ddf95b96d9300f295"><img src="https://joss.theoj.org/papers/ee73bb825d396e5ddf95b96d9300f295/status.svg"></a>
Markdown: [![status](https://joss.theoj.org/papers/ee73bb825d396e5ddf95b96d9300f295/status.svg)](https://joss.theoj.org/papers/ee73bb825d396e5ddf95b96d9300f295)

Reviewers and authors:

Please avoid lengthy details of difficulties in the review thread. Instead, please create a new issue in the target repository and link to those issues (especially acceptance-blockers) by leaving comments in the review thread below. (For completists: if the target issue tracker is also on GitHub, linking the review thread in the issue or vice versa will create corresponding breadcrumb trails in the link target.)

Reviewer instructions & questions

@zingale & @rtfisher, please carry out your review in this issue by updating the checklist below. If you cannot edit the checklist please:

Make sure you're logged in to your GitHub account
Be sure to accept the invite at this URL: https://github.com/openjournals/joss-reviews/invitations

The reviewer guidelines are available here: https://joss.readthedocs.io/en/latest/reviewer_guidelines.html. Any questions/concerns please let @danielskatz know.

✨ Please try and complete your review in the next two weeks ✨

Review checklist for @zingale

Conflict of interest

[x] As the reviewer I confirm that I have read the JOSS conflict of interest policy and that there are no conflicts of interest for me to review this work.

Code of Conduct

[x] I confirm that I read and will adhere to the JOSS code of conduct.

General checks

[x] Repository: Is the source code for this software available at the repository url?
[x] License: Does the repository contain a plain-text LICENSE file with the contents of an OSI approved software license?
[x] Version: v2.6.1
[x] Authorship: Has the submitting author (@bwoshea) made major contributions to the software? Does the full list of paper authors seem appropriate and complete?

Functionality

[x] Installation: Does installation proceed as outlined in the documentation?
[x] Functionality: Have the functional claims of the software been confirmed?
[x] Performance: If there are any performance claims of the software, have they been confirmed? (If there are no claims, please check off this item.)

Documentation

[x] A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
[x] Installation instructions: Is there a clearly-stated list of dependencies? Ideally these should be handled with an automated package management solution.
[x] Example usage: Do the authors include examples of how to use the software (ideally to solve real-world analysis problems).
[x] Functionality documentation: Is the core functionality of the software documented to a satisfactory level (e.g., API method documentation)?
[x] Automated tests: Are there automated tests or manual steps described so that the function of the software can be verified?
[x] Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support

Software paper

[x] Authors: Does the paper.md file include a list of authors with their affiliations?
[x] A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
[x] References: Do all archival references that should have a DOI list one (e.g., papers, datasets, software)?

Review checklist for @rtfisher

Conflict of interest

[x] As the reviewer I confirm that I have read the JOSS conflict of interest policy and that there are no conflicts of interest for me to review this work.

Code of Conduct

[x] I confirm that I read and will adhere to the JOSS code of conduct.

General checks

[x] Repository: Is the source code for this software available at the repository url?
[x] License: Does the repository contain a plain-text LICENSE file with the contents of an OSI approved software license?
[x] Version: v2.6.1
[x] Authorship: Has the submitting author (@bwoshea) made major contributions to the software? Does the full list of paper authors seem appropriate and complete?

Functionality

[x] Installation: Does installation proceed as outlined in the documentation?
[x] Functionality: Have the functional claims of the software been confirmed?
[x] Performance: If there are any performance claims of the software, have they been confirmed? (If there are no claims, please check off this item.)

Documentation

[x] A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
[x] Installation instructions: Is there a clearly-stated list of dependencies? Ideally these should be handled with an automated package management solution.
[x] Example usage: Do the authors include examples of how to use the software (ideally to solve real-world analysis problems).
[x] Functionality documentation: Is the core functionality of the software documented to a satisfactory level (e.g., API method documentation)?
[x] Automated tests: Are there automated tests or manual steps described so that the function of the software can be verified?
[x] Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support

Software paper

[x] Authors: Does the paper.md file include a list of authors with their affiliations?
[x] A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
[x] References: Do all archival references that should have a DOI list one (e.g., papers, datasets, software)?

accepted published recommend-accept review

Source

whedon

All 74 comments

Hello human, I'm @whedon, a robot that can help you with some common editorial tasks. @zingale, @rtfisher it looks like you're currently assigned to review this paper :tada:.

:star: Important :star:

If you haven't already, you should seriously consider unsubscribing from GitHub notifications for this (https://github.com/openjournals/joss-reviews) repository. As a reviewer, you're probably currently watching this repository which means for GitHub's default behaviour you will receive notifications (emails) for all reviews 😿

To fix this do the following two things:

Set yourself as 'Not watching' https://github.com/openjournals/joss-reviews:

watching

You may also like to change your default settings for this watching repositories in your GitHub profile here: https://github.com/settings/notifications

notifications

For a list of things I can do to help you, just type:

@whedon commands

For example, to regenerate the paper pdf after making changes in the paper's md or bib files, type:

@whedon generate pdf

whedon on 9 Aug 2019

Attempting PDF compilation. Reticulating splines etc...

whedon on 9 Aug 2019

:point_right: Check article proof :page_facing_up: :point_left:

whedon on 9 Aug 2019

I had to update the repository URL on the top comment of this issue. (note to @arfon - whedon didn't use the URL in the pre-report issue, and this may also lead to a problem when we do the final acceptance.)

danielskatz on 9 Aug 2019

@whedon generate pdf

danielskatz on 9 Aug 2019

Attempting PDF compilation. Reticulating splines etc...

whedon on 9 Aug 2019

:point_right: Check article proof :page_facing_up: :point_left:

whedon on 9 Aug 2019

@bwoshea - merging https://github.com/enzo-project/enzo-dev/pull/93 will fix 2 small bib entries

danielskatz on 9 Aug 2019

👋 @zingale, @rtfisher - we are ready for you to review this now. Please read the first 2 comments in this issue carefully. Your job now is to go through the paper (the latest "check article proof" in this issue) and the repository, and check off items in your checklist until you are done, and if you find problems, either report them here or create an issue in the repo and mention this issue in that one.

If you have any questions or problems, please let me know.

danielskatz on 9 Aug 2019

I am just beginning the review process. I find the documentation link (https://enzo-project.org/docs/2.6/) under https://github.com/enzo-project/enzo-dev returns a 404.

rtfisher on 10 Aug 2019

@bwoshea - merging enzo-project/enzo-dev#93 will fix 2 small bib entries

Now that this has been done, I will recompile - @bwoshea, you can do this too, just the way I am, with an instruction in whedon in a new comment (letting you know in case more changes are needed)

danielskatz on 11 Aug 2019

👍1

@whedon generate pdf

danielskatz on 11 Aug 2019

Attempting PDF compilation. Reticulating splines etc...

whedon on 11 Aug 2019

:point_right: Check article proof :page_facing_up: :point_left:

whedon on 11 Aug 2019

regarding

References: Do all archival references that should have a DOI list one (e.g., papers, datasets, software)

There are 2 references to Enzo itself, referring to github. Should these be changed to an archived version of the code with a DOI?

zingale on 11 Aug 2019

I'm done with my pass through the docs and have filed a number of issues, some which I would like to see addressed for the review (marked with [JOSS REVIEW]) and others that are suggestions to take into consideration.

I will try running some standard test problems next.

zingale on 14 Aug 2019

@zingale - thanks for all your comments so far. If possible, when you open an issue in https://github.com/enzo-project/enzo-dev, just mention this review thread (https://github.com/openjournals/joss-reviews/issues/1636) in that issue rather than posted that issue in this thread. This will insert a note here that will show if the issue is open or closed.

danielskatz on 15 Aug 2019

oh, I see, I did it backwards. I'll fix that.

zingale on 16 Aug 2019

👋 @bwoshea - note that we're waiting for you to respond to a bunch of open issues - listed above...

danielskatz on 26 Aug 2019

Just to clarify, I am still in the process of completing my review. @bwoshea indicates that the ENZO team is awaiting this completion prior to starting their responses.

rtfisher on 26 Aug 2019

That's ok, but JOSS reviews typically are more interactive. Maybe with a complex team like this, it makes sense to wait.

danielskatz on 26 Aug 2019

@danielskatz Sorry about that. Given the vigor of @zingale 's review and the distributed nature of the Enzo team, we would prefer to wait until the reviews are completed (or at least the first wave of feedback) until we proceed. If you insist on doing it otherwise I'm sure we can make that happen, though.

bwoshea on 29 Aug 2019

It’s fine to wait, as I said previously

danielskatz on 29 Aug 2019

Ok, I've completed my initial review. It looks like I also identified some of the same issues as @zingale, though our final assessments of the JOSS requirements differ slightly.

rtfisher on 29 Aug 2019

Thanks @rtfisher! Now over to @bwoshea and the Enzo team...

danielskatz on 30 Aug 2019

@danielskatz We're on it! @rtfisher and @zingale , thank you for all of your feedback!

bwoshea on 8 Sep 2019

@danielskatz @rtfisher @zingale The Enzo developers have been working on the issues you raised (at https://github.com/enzo-project/enzo-dev/issues), and we currently have a bunch of open pull requests to address them (at https://github.com/enzo-project/enzo-dev/pulls). Some of the issues have comments on them that directly address reviewer comments without a PR (e.g., https://github.com/enzo-project/enzo-dev/issues/108), but mostly the changes in the PRs themselves should address your comments.

Please let us know how you'd like to proceed at this point!

bwoshea on 12 Sep 2019

@bwoshea - it's up to you, with the goal of getting the reviewers to agree that the submission meets the criteria. You could make all the changes, then ask the reviewers to check them, or ask reviewers for specific opinions on possible changes first.

danielskatz on 12 Sep 2019

I would suggest that you refer to the PRs in the issues we raised and when we are satisfied, we will close the issues that we opened for the review, and that will be reflected here automatically.

zingale on 12 Sep 2019

@danielskatz -- duly noted, thank you! We'll engage the reviewers on the open PRs rather than waiting. That way we'll be able to respond more quickly.

@zingale -- Great suggestion, thanks. I've done that now! We have a couple of outstanding issues that still need developer attention, but the bulk have been addressed.

bwoshea on 12 Sep 2019

👍1

@zingale @rtfisher just to give you a heads-up, we've addressed all of the issues you have raised but one, and that should be done in the next day or so. We've left all off the PRs open so that you can see what has been changed, and all PRs and issues are cross-referenced. We're ready for your feedback whenever you are available.

Edit: we now (as of 2019-09-15) have PRs that address all issues raised!

bwoshea on 13 Sep 2019

It looks like all of my issues have been addressed via PRs. Once those PRs are merged, I am happy with accepting.

zingale on 16 Sep 2019

I am fine with accepting after merger as well.

rtfisher on 17 Sep 2019

@zingale @rtfisher thanks!

@rtfisher to clarify, there are two open issues that you raised (https://github.com/enzo-project/enzo-dev/issues/108 and https://github.com/enzo-project/enzo-dev/issues/109 ) - would you mind making a quick note saying you're OK with what we've done for those?

bwoshea on 17 Sep 2019

Thank you, @bwoshea. Sorry for the delayed response -- I was serving on a panel today.

I concur that the issues are now resolved, and have closed both.

rtfisher on 18 Sep 2019

The next steps will be

The authors (@bwoshea) to make any remaining changes (sorry, I can't easily tell if there's more to do or if you've already done this because the PRs didn't reference this review thread) and saying that they are done here.
The reviewers (@zingale and @rtfisher) to confirm that they are satisfied by checking off the remaining items in their review checklists above and by saying so in this thread.

danielskatz on 18 Sep 2019

@danielskatz we have made the remaining changes!

bwoshea on 19 Sep 2019

👍1

👋 @zingale and @rtfisher: please confirm that you are satisfied by checking off the remaining items in your review checklists above and by saying so in this thread. If not, please say what you think any current issues are.