Joss-reviews: [REVIEW]: The Pulsar Signal Simulator: A Python package for simulating radio signal data from pulsars

[whedon]

Submitting author: @Hazboun6 (Jeffrey Hazboun)
Repository: https://github.com/PsrSigSim/PsrSigSim
Version: v1.0.0
Editor: @mbobra
Reviewer: @JBorrow, @arjunsavel
Archive: Pending

:warning: JOSS reduced service mode :warning:

Due to the challenges of the COVID-19 pandemic, JOSS is currently operating in a "reduced service mode". You can read more about what that means in our blog post.

Status

Status badge code:

HTML: <a href="https://joss.theoj.org/papers/6f2caac6394c899d461075963b5d7e45"><img src="https://joss.theoj.org/papers/6f2caac6394c899d461075963b5d7e45/status.svg"></a>
Markdown: [![status](https://joss.theoj.org/papers/6f2caac6394c899d461075963b5d7e45/status.svg)](https://joss.theoj.org/papers/6f2caac6394c899d461075963b5d7e45)

Reviewers and authors:

Please avoid lengthy details of difficulties in the review thread. Instead, please create a new issue in the target repository and link to those issues (especially acceptance-blockers) by leaving comments in the review thread below. (For completists: if the target issue tracker is also on GitHub, linking the review thread in the issue or vice versa will create corresponding breadcrumb trails in the link target.)

Reviewer instructions & questions

@JBorrow & @arjunsavel, please carry out your review in this issue by updating the checklist below. If you cannot edit the checklist please:

1. Make sure you're logged in to your GitHub account
2. Be sure to accept the invite at this URL: https://github.com/openjournals/joss-reviews/invitations

The reviewer guidelines are available here: https://joss.readthedocs.io/en/latest/reviewer_guidelines.html. Any questions/concerns please let @mbobra know.

✨ Please start on your review when you are able, and be sure to complete your review in the next six weeks, at the very latest ✨

Review checklist for @JBorrow

Conflict of interest

• [x] I confirm that I have read the JOSS conflict of interest (COI) policy and that: I have no COIs with reviewing this work or that any perceived COIs have been waived by JOSS for the purpose of this review.

Code of Conduct

• [x] I confirm that I read and will adhere to the JOSS code of conduct.

General checks

• [x] Repository: Is the source code for this software available at the repository url?
• [x] License: Does the repository contain a plain-text LICENSE file with the contents of an OSI approved software license?
• [x] Contribution and authorship: Has the submitting author (@Hazboun6) made major contributions to the software? Does the full list of paper authors seem appropriate and complete?
• [x] Substantial scholarly effort: Does this submission meet the scope eligibility described in the JOSS guidelines

Functionality

• [ ] Installation: Does installation proceed as outlined in the documentation?
• [x] Functionality: Have the functional claims of the software been confirmed?
• [x] Performance: If there are any performance claims of the software, have they been confirmed? (If there are no claims, please check off this item.)

Documentation

• [ ] A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
• [ ] Installation instructions: Is there a clearly-stated list of dependencies? Ideally these should be handled with an automated package management solution.
• [ ] Example usage: Do the authors include examples of how to use the software (ideally to solve real-world analysis problems).
• [ ] Functionality documentation: Is the core functionality of the software documented to a satisfactory level (e.g., API method documentation)?
• [ ] Automated tests: Are there automated tests or manual steps described so that the functionality of the software can be verified?
• [x] Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support

Software paper

• [x] Summary: Has a clear description of the high-level functionality and purpose of the software for a diverse, non-specialist audience been provided?
• [ ] A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
• [ ] State of the field: Do the authors describe how this software compares to other commonly-used packages?
• [ ] Quality of writing: Is the paper well written (i.e., it does not require editing for structure, language, or writing quality)?
• [ ] References: Is the list of references complete, and is everything cited appropriately that should be cited (e.g., papers, datasets, software)? Do references in the text use the proper citation syntax?

Review checklist for @arjunsavel

Conflict of interest

• [x] I confirm that I have read the JOSS conflict of interest (COI) policy and that: I have no COIs with reviewing this work or that any perceived COIs have been waived by JOSS for the purpose of this review.

Code of Conduct

• [x] I confirm that I read and will adhere to the JOSS code of conduct.

General checks

• [x] Repository: Is the source code for this software available at the repository url?
• [x] License: Does the repository contain a plain-text LICENSE file with the contents of an OSI approved software license?
• [ ] Contribution and authorship: Has the submitting author (@Hazboun6) made major contributions to the software? Does the full list of paper authors seem appropriate and complete?
• [x] Substantial scholarly effort: Does this submission meet the scope eligibility described in the JOSS guidelines

Functionality

• [ ] Installation: Does installation proceed as outlined in the documentation?
• [x] Functionality: Have the functional claims of the software been confirmed?
• [x] Performance: If there are any performance claims of the software, have they been confirmed? (If there are no claims, please check off this item.)

Documentation

• [x] A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
• [x] Installation instructions: Is there a clearly-stated list of dependencies? Ideally these should be handled with an automated package management solution.
• [x] Example usage: Do the authors include examples of how to use the software (ideally to solve real-world analysis problems).
• [x] Functionality documentation: Is the core functionality of the software documented to a satisfactory level (e.g., API method documentation)?
• [x] Automated tests: Are there automated tests or manual steps described so that the functionality of the software can be verified?
• [x] Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support

Software paper

• [x] Summary: Has a clear description of the high-level functionality and purpose of the software for a diverse, non-specialist audience been provided?
• [x] A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
• [ ] State of the field: Do the authors describe how this software compares to other commonly-used packages?
• [ ] Quality of writing: Is the paper well written (i.e., it does not require editing for structure, language, or writing quality)?
• [ ] References: Is the list of references complete, and is everything cited appropriately that should be cited (e.g., papers, datasets, software)? Do references in the text use the proper citation syntax?

[whedon]

Hello human, I'm @whedon, a robot that can help you with some common editorial tasks. @JBorrow, @arjunsavel it looks like you're currently assigned to review this paper :tada:.

:warning: JOSS reduced service mode :warning:

Due to the challenges of the COVID-19 pandemic, JOSS is currently operating in a "reduced service mode". You can read more about what that means in our blog post.

:star: Important :star:

If you haven't already, you should seriously consider unsubscribing from GitHub notifications for this (https://github.com/openjournals/joss-reviews) repository. As a reviewer, you're probably currently watching this repository which means for GitHub's default behaviour you will receive notifications (emails) for all reviews 😿

To fix this do the following two things:

1. Set yourself as 'Not watching' https://github.com/openjournals/joss-reviews:

1. You may also like to change your default settings for this watching repositories in your GitHub profile here: https://github.com/settings/notifications

For a list of things I can do to help you, just type:

@whedon commands

For example, to regenerate the paper pdf after making changes in the paper's md or bib files, type:

@whedon generate pdf

[whedon]

Reference check summary (note 'MISSING' DOIs are suggestions that need verification):

OK DOIs

- 10.1038/217709a0 is OK
- 10.1109/MCSE.2011.37 is OK
- 10.3847/1538-3881/aabc4f is OK
- 10.1007/s00159-019-0115-7 is OK
- 10.1093/nsr/nwx126 is OK
- 10.1086/181708 is OK
- 10.1086/159690 is OK
- 10.5281/zenodo.3756164 is OK
- 10.1017/pasa.2020.19 is OK

MISSING DOIs

- None

INVALID DOIs

- None

[whedon]

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

[mbobra]

👋 @JBorrow @arjunsavel Thank you so much for agreeing to review this submission! You'll see your reviewer checklists above. Please feel free to ask questions here on this review thread.

Please note that this article is in collaboration with AAS publishing (the submitting authors submitted a companion paper to The Astrophysical Journal). As part of this collaboration, AAS publishing makes a small donation to the running costs of JOSS. Please take a look at the blog posts about the AAS-JOSS collaboration and JOSS' financial model. Please let me know if you have any issues with this.

[JBorrow]

The authors have presented a new package for the simulation and modelling of pulsar observations. It has a clean object-oriented interface, and exceptionally detailed narrative documentation and tutorial content. I found the interface particularly intuitive, and it was great (as somebody who is a 'pure' simulator) to play around with the different telescopes. I can see clearly both the scientific as well as educational benefit of such a package. As somebody not directly within the field of radio astronomy, I cannot comment on the scientific merit of the package or its results, but I will leave this to those reviewing the ApJ paper.

I have opened the following issues within the main repository:

• https://github.com/PsrSigSim/PsrSigSim/issues/156
• https://github.com/PsrSigSim/PsrSigSim/issues/157
• https://github.com/PsrSigSim/PsrSigSim/issues/158
• https://github.com/PsrSigSim/PsrSigSim/issues/159
• https://github.com/PsrSigSim/PsrSigSim/issues/160

with various concerns. Most of these are minor and should be a relatively quick fix (e.g. typos, adding a section to the paper looking at other software packages in the field). My one major concern, though, is the large amount of warnings produced when running the examples and tests. In particular, when running the test suite, I saw over 1000 warnings of various types (Runtime warnings from taking negative square roots and deprecation warnings). I would like to have these resolved before being confident in accepting this package, particularly as they are so numerous.

As an additional (not necessary) comment, I think that this package would really benefit from an integration with a symbolic unit library such as unyt, pint, or astropy.units. It appears that there has been some effort to use this within the i/o subsystem, but for the user-facing API knowing the input and output units would be really nice.

[arjunsavel]

I've completed my review:

With PsrSigSim, the authors have provided a well-documented and well-organized package for simulating pulsar observations. I don't specialize in radio astronomy, but to the extent of my knowledge, I believe it to be of scientific merit.

Of the issues that I have opened in PsrSigSim repository, the below are relevant for acceptance. Many are related to grammar and consistency.

• https://github.com/PsrSigSim/PsrSigSim/issues/164
• https://github.com/PsrSigSim/PsrSigSim/issues/163
• https://github.com/PsrSigSim/PsrSigSim/issues/162
• https://github.com/PsrSigSim/PsrSigSim/issues/161

For https://github.com/PsrSigSim/PsrSigSim/issues/164, a full implementation of a user-input random seed would not be necessary for acceptance; however, I would recommend briefly addressing effects of randomness on the documentation plots.

I reproduced the errors (and fixes) on installation/testing that @JBorrow reported above (running Python 3.7.6).

I also wanted to touch base about the authorship list — I noticed that a few people are listed in the paper but not in AUTHORS.RST (Nathan Garver-Daniels, Maura A. McLaughlin).

Overall, the package looks great to me, and I'll be signed off pending the resolution of the above issues! Looking forward to the companion ApJ paper.

[Hazboun6]

Thank you @JBorrow and @arjunsavel for your speedy reviews! We're looking over the Issues you both posted now and we'll let you know when we've been able to attend to them. Hopefully we'll be able to make most changes this week.

[Hazboun6]

@mbobra One point that @JBorrow made in Issue 160 was about a reference to the AAS paper to which this work is linked. Is there a JOSS standard for how you'd like us to refer to that paper? I'm imagining it will probably take longer to appear/publish. Would you like it as .bib entry? Currently it is written as [Shapiro-Albert,2020], but I just realized that if I was following the AAS style it should be [Shapiro-Albert, In Prep], which is probably what led to some confusion.

[mbobra]

@Hazboun6 Good question. You don't need to cite the AAS paper. The DOI for the AAS paper will appear the JOSS paper automatically (see examples here and here; look at the left margin).

There is some documentation about the AAS publishing process process here. Here is the order of operations:

1. The JOSS reviewers agree that the software package PsrSigSim passes review.
2. Your paper is accepted by JOSS. However, we don't publish it yet. We pause the publication of the review pending the DOI from ApJ. Practically speaking, pausing the review means I add a label to this review thread that says "paused" and we leave this issue open.
3. Your paper is accepted into ApJ. We the publish the JOSS paper and automatically include the ApJ DOI, which automatically closes this issue.

If you do want to specifically cite the ApJ paper in your JOSS paper, you can wait until we're at Step 3, modify your paper.md file to include the complete citation, and then we can publish the JOSS paper.