Joss-reviews: [REVIEW]: PyGDH ("pigged"): Python Grid Discretization Helper

[whedon]

Submitting author: @kfhiga (Kenneth Higa)
Repository: https://bitbucket.org/berkeleylab/esdr-pygdh/
Version: 0.4.1
Editor: @melissawm
Reviewer: @JamieJQuinn, @zhaowei0566
Archive: Pending

:warning: JOSS reduced service mode :warning:

Due to the challenges of the COVID-19 pandemic, JOSS is currently operating in a "reduced service mode". You can read more about what that means in our blog post.

Status

Status badge code:

HTML: <a href="https://joss.theoj.org/papers/a5c27a933efb1e4dfa308059de26d0f2"><img src="https://joss.theoj.org/papers/a5c27a933efb1e4dfa308059de26d0f2/status.svg"></a>
Markdown: [![status](https://joss.theoj.org/papers/a5c27a933efb1e4dfa308059de26d0f2/status.svg)](https://joss.theoj.org/papers/a5c27a933efb1e4dfa308059de26d0f2)

Reviewers and authors:

Please avoid lengthy details of difficulties in the review thread. Instead, please create a new issue in the target repository and link to those issues (especially acceptance-blockers) by leaving comments in the review thread below. (For completists: if the target issue tracker is also on GitHub, linking the review thread in the issue or vice versa will create corresponding breadcrumb trails in the link target.)

Reviewer instructions & questions

@JamieJQuinn & @zhaowei0566, please carry out your review in this issue by updating the checklist below. If you cannot edit the checklist please:

1. Make sure you're logged in to your GitHub account
2. Be sure to accept the invite at this URL: https://github.com/openjournals/joss-reviews/invitations

The reviewer guidelines are available here: https://joss.readthedocs.io/en/latest/reviewer_guidelines.html. Any questions/concerns please let @melissawm know.

✨ Please start on your review when you are able, and be sure to complete your review in the next six weeks, at the very latest ✨

Review checklist for @JamieJQuinn

Conflict of interest

• [x] I confirm that I have read the JOSS conflict of interest (COI) policy and that: I have no COIs with reviewing this work or that any perceived COIs have been waived by JOSS for the purpose of this review.

Code of Conduct

• [x] I confirm that I read and will adhere to the JOSS code of conduct.

General checks

• [x] Repository: Is the source code for this software available at the repository url?
• [x] License: Does the repository contain a plain-text LICENSE file with the contents of an OSI approved software license?
• [x] Contribution and authorship: Has the submitting author (@kfhiga) made major contributions to the software? Does the full list of paper authors seem appropriate and complete?
• [x] Substantial scholarly effort: Does this submission meet the scope eligibility described in the JOSS guidelines

Functionality

• [x] Installation: Does installation proceed as outlined in the documentation?
• [x] Functionality: Have the functional claims of the software been confirmed?
• [x] Performance: If there are any performance claims of the software, have they been confirmed? (If there are no claims, please check off this item.)

Documentation

• [x] A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
• [x] Installation instructions: Is there a clearly-stated list of dependencies? Ideally these should be handled with an automated package management solution.
• [x] Example usage: Do the authors include examples of how to use the software (ideally to solve real-world analysis problems).
• [x] Functionality documentation: Is the core functionality of the software documented to a satisfactory level (e.g., API method documentation)?
• [x] Automated tests: Are there automated tests or manual steps described so that the functionality of the software can be verified?
• [x] Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support

Software paper

• [x] Summary: Has a clear description of the high-level functionality and purpose of the software for a diverse, non-specialist audience been provided?
• [x] A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
• [x] State of the field: Do the authors describe how this software compares to other commonly-used packages?
• [x] Quality of writing: Is the paper well written (i.e., it does not require editing for structure, language, or writing quality)?
• [x] References: Is the list of references complete, and is everything cited appropriately that should be cited (e.g., papers, datasets, software)? Do references in the text use the proper citation syntax?

Review checklist for @zhaowei0566

Conflict of interest

• [x] I confirm that I have read the JOSS conflict of interest (COI) policy and that: I have no COIs with reviewing this work or that any perceived COIs have been waived by JOSS for the purpose of this review.

Code of Conduct

• [x] I confirm that I read and will adhere to the JOSS code of conduct.

General checks

• [ ] Repository: Is the source code for this software available at the repository url?
• [ ] License: Does the repository contain a plain-text LICENSE file with the contents of an OSI approved software license?
• [ ] Contribution and authorship: Has the submitting author (@kfhiga) made major contributions to the software? Does the full list of paper authors seem appropriate and complete?
• [ ] Substantial scholarly effort: Does this submission meet the scope eligibility described in the JOSS guidelines

Functionality

• [ ] Installation: Does installation proceed as outlined in the documentation?
• [ ] Functionality: Have the functional claims of the software been confirmed?
• [ ] Performance: If there are any performance claims of the software, have they been confirmed? (If there are no claims, please check off this item.)

Documentation

• [ ] A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
• [ ] Installation instructions: Is there a clearly-stated list of dependencies? Ideally these should be handled with an automated package management solution.
• [ ] Example usage: Do the authors include examples of how to use the software (ideally to solve real-world analysis problems).
• [ ] Functionality documentation: Is the core functionality of the software documented to a satisfactory level (e.g., API method documentation)?
• [ ] Automated tests: Are there automated tests or manual steps described so that the functionality of the software can be verified?
• [ ] Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support

Software paper

• [ ] Summary: Has a clear description of the high-level functionality and purpose of the software for a diverse, non-specialist audience been provided?
• [ ] A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
• [ ] State of the field: Do the authors describe how this software compares to other commonly-used packages?
• [ ] Quality of writing: Is the paper well written (i.e., it does not require editing for structure, language, or writing quality)?
• [ ] References: Is the list of references complete, and is everything cited appropriately that should be cited (e.g., papers, datasets, software)? Do references in the text use the proper citation syntax?

[whedon]

Hello human, I'm @whedon, a robot that can help you with some common editorial tasks. @JamieJQuinn, @zhaowei0566 it looks like you're currently assigned to review this paper :tada:.

:warning: JOSS reduced service mode :warning:

Due to the challenges of the COVID-19 pandemic, JOSS is currently operating in a "reduced service mode". You can read more about what that means in our blog post.

:star: Important :star:

If you haven't already, you should seriously consider unsubscribing from GitHub notifications for this (https://github.com/openjournals/joss-reviews) repository. As a reviewer, you're probably currently watching this repository which means for GitHub's default behaviour you will receive notifications (emails) for all reviews 😿

To fix this do the following two things:

1. Set yourself as 'Not watching' https://github.com/openjournals/joss-reviews:

1. You may also like to change your default settings for this watching repositories in your GitHub profile here: https://github.com/settings/notifications

For a list of things I can do to help you, just type:

@whedon commands

For example, to regenerate the paper pdf after making changes in the paper's md or bib files, type:

@whedon generate pdf

[whedon]

Reference check summary (note 'MISSING' DOIs are suggestions that need verification):

OK DOIs

- 10.11588/ans.2015.100.20553 is OK
- 10.1515/jnum-2012-0013 is OK
- 10.1109/MCSE.2009.52 is OK
- 10.1109/mcse.2011.37 is OK
- 10.1038/s41592-019-0686-2 is OK
- 10.1109/MCSE.2007.55 is OK
- 10.1109/MCSE.2010.118 is OK
- 10.1149/2.0091507jes is OK

MISSING DOIs

- None

INVALID DOIs

- None

[whedon]

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

[melissawm]

👋🏼 @kfhiga @jamiejquinn @zhaowei0566 this is the review thread for the paper. All of our communications will happen here from now on.

Both reviewers have checklists at the top of this thread with the JOSS requirements. As you go over the submission, please check any items that you feel have been satisfied. There are also links to the JOSS reviewer guidelines.

The JOSS review is different from most other journals. Our goal is to work with the authors to help them meet our criteria instead of merely passing judgment on the submission. As such, the reviewers are encouraged to submit issues and pull requests on the software repository. When doing so, please mention openjournals/joss-reviews#2744 so that a link is created to this thread (and I can keep an eye on what is happening). Please also feel free to comment and ask questions on this thread. In my experience, it is better to post comments/questions/suggestions as you come across them instead of waiting until you've reviewed the entire package.

We aim for reviews to be completed within about 2-4 weeks. Please let me know if any of you require some more time. We can also use Whedon (our bot) to set automatic reminders if you know you'll be away for a known period of time.

Please feel free to ping me (@melissawm) if you have any questions/concerns.

[kfhiga]

@melissawm https://github.com/melissawm @JamieJQuinn
https://github.com/JamieJQuinn @zhaowei0566
https://github.com/zhaowei0566 Thank you all for your time!
Kenny

On Mon, Oct 12, 2020 at 3:25 PM Melissa Weber Mendonça <
[email protected]> wrote:

👋🏼 @kfhiga https://github.com/kfhiga @JamieJQuinn
https://github.com/JamieJQuinn @zhaowei0566
https://github.com/zhaowei0566 this is the review thread for the paper.
All of our communications will happen here from now on.

Both reviewers have checklists at the top of this thread with the JOSS
requirements. As you go over the submission, please check any items that
you feel have been satisfied. There are also links to the JOSS reviewer
guidelines.

The JOSS review is different from most other journals. Our goal is to work
with the authors to help them meet our criteria instead of merely passing
judgment on the submission. As such, the reviewers are encouraged to submit
issues and pull requests on the software repository. When doing so, please
mention openjournals/joss-reviews#2744 so that a link is created to this
thread (and I can keep an eye on what is happening). Please also feel free
to comment and ask questions on this thread. In my experience, it is better
to post comments/questions/suggestions as you come across them instead of
waiting until you've reviewed the entire package.

We aim for reviews to be completed within about 2-4 weeks. Please let me
know if any of you require some more time. We can also use Whedon (our bot)
to set automatic reminders if you know you'll be away for a known period of
time.

Please feel free to ping me (@melissawm https://github.com/melissawm)
if you have any questions/concerns.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/openjournals/joss-reviews/issues/2744#issuecomment-707375618,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AHRT54WHEPQG5UIHUBXHIA3SKN655ANCNFSM4SNP3ELA
.

[JamieJQuinn]

Repo is on Bitbucket so couldn't link this repo directly. Added a few issues about installation on non-Debian systems.

[melissawm]

:wave: Hello all, just checking in to see if there's anything I can help you with.

[kfhiga]

Hi everyone and thanks for checking @melissawm. I've committed some changes
in response to the issues noted in the Bitbucket issue tracker. I should
probably have mentioned that here in case Bitbucket doesn't send
notifications on issue updates (sorry, still pretty new to this).

Thanks,
Kenny

On Wed, Oct 28, 2020 at 10:52 AM Melissa Weber Mendonça <
[email protected]> wrote:

👋 Hello all, just checking in to see if there's anything I can help you
with.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/openjournals/joss-reviews/issues/2744#issuecomment-718105457,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AHRT54UTAMIN3P74CRDVJFLSNBK7HANCNFSM4SNP3ELA
.

[melissawm]

No problem! This is the first time I edit a submission that is not on github, so I'm new to this workflow as well. Glad to see things are ok, please ping me if there's any questions or comments. Thanks!

[JamieJQuinn]

I'm happy to accept this with a few changes, mostly to the documentation:

Documentation:

• I found the SPHINX annotations distracting; I imagine they would be very confusing to a new python user. Can these be removed from the examples in the documentation? I think they're fine in the actual example code.
• I don't think the full source code for each example is required in the documentation. It's easy enough for a user to grab the examples from the repo. Additionally, it may help to shorten the length of the documentation file (which may be intimidating for new users). Could the full example code be removed from the end of each section?
• There's a statement at the end of page one of the documentation:
  "Users should be warned that although PyGDH has been successfully used in past and ongoing research products andis anticipated to be useful to others, it is still under development and should not be considered to be well-tested, stablesoftware."
  I would expect some users to read this and be unsure about whether the results of code using pygdh can be trusted, however you've stated that the numpy solvers are used, which are very mature and well-tested. Additionally, you provide a number of examples with analytical comparisons so in my opinion, the code is well validated, if not widely verified through community use. Could this statement be reworded to reflect that the code has been validated with test problems and uses a mature solver under the hood?
• I'd suggest changing section 3.3.1 so that it doesn't look so much like a full example. I personally followed that as if it were a functioning example only to find out there were parts missing. Perhaps this section could instead just describe the program structure without examples and then use the working example in 3.4 to illustrate those. The initial paragraph in 3.3.1 is sufficient in my opinion.

Code:

• The examples folder is very busy due to all the example code & outputs being in one single folder. Can these be split into one folder per example?
• I like the level of detail in what the code prints as it runs however it could be clearer what is an info statement, a warning, or an error. Could the difference between info, warnings and errors be made clearer, e.g. replacing statements like '! example error" with "ERROR: example error" and "INFO: creating domain"?

Otherwise I think this is a great package for an domain scientist who just wants some results without getting too deep into the nitty-gritty. As much as I feel I've torn apart the documentation (sorry!), it's a fantastic introduction for a new user, both to the software and to solving these kind of problems numerically. Similarly, the breadth of the provided examples in impressive and they seem to provide a comprehensive view of the problems solvable with pygdh.

[kfhiga]

Thank you for the detailed advice! I will get to work on these and let you
know when I've made the changes.
Kenny

On Sat, Nov 14, 2020 at 5:32 AM Jamie J Quinn [e-mail]
wrote:

I'm happy to accept this with a few changes, mostly to the documentation:

Documentation:

• I found the SPHINX annotations distracting; I imagine they would be
  very confusing to a new python user. Can these be removed from the examples
  in the documentation? I think they're fine in the actual example code.
• I don't think the full source code for each example is required in
  the documentation. It's easy enough for a user to grab the examples from
  the repo. Additionally, it may help to shorten the length of the
  documentation file (which may be intimidating for new users). Could the
  full example code be removed from the end of each section?
• There's a statement at the end of page one of the documentation:
  "Users should be warned that although PyGDH has been successfully
  used in past and ongoing research products andis anticipated to be useful
  to others, it is still under development and should not be considered to be
  well-tested, stablesoftware."
  I would expect some users to read this and be unsure about whether the
  results of code using pygdh can be trusted, however you've stated that the
  numpy solvers are used, which are very mature and well-tested.
  Additionally, you provide a number of examples with analytical comparisons
  so in my opinion, the code is well validated, if not widely verified
  through community use. Could this statement be reworded to reflect that the
  code has been validated with test problems and uses a mature solver under
  the hood?
• I'd suggest changing section 3.3.1 so that it doesn't look so much
  like a full example. I personally followed that as if it were a functioning
  example only to find out there were parts missing. Perhaps this section
  could instead just describe the program structure without examples and then
  use the working example in 3.4 to illustrate those. The initial paragraph
  in 3.3.1 is sufficient in my opinion.

Code:

• The examples folder is very busy due to all the example code &
  outputs being in one single folder. Can these be split into one folder per
  example?
• I like the level of detail in what the code prints as it runs
  however it could be clearer what is an info statement, a warning, or an
  error. Could the difference between info, warnings and errors be made
  clearer, e.g. replacing statements like '! example error" with "ERROR:
  example error" and "INFO: creating domain"?

Otherwise I think this is a great package for an domain scientist who just
wants some results without getting too deep into the nitty-gritty. As much
as I feel I've torn apart the documentation (sorry!), it's a fantastic
introduction for a new user, both to the software and to solving these kind
of problems numerically. Similarly, the breadth of the provided examples in
impressive and they seem to provide a comprehensive view of the problems
solvable with pygdh.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/openjournals/joss-reviews/issues/2744#issuecomment-727207801,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AHRT54RMK6SU2VHWSRQ7GXLSP2BFNANCNFSM4SNP3ELA
.