Joss-reviews: [REVIEW]: Prism: Multiple spline regression with regularization, dimensionality reduction, and feature selection

[whedon]

Submitting author: @cMadan (Christopher R Madan)
Repository: https://github.com/cMadan/prism
Version: v2.0.0
Editor: @arfon
Reviewer: @Kevin-Mattheus-Moerman
Archive: http://dx.doi.org/10.5281/zenodo.56821

Status

Status badge code:

HTML: <a href="http://joss.theoj.org/papers/c73c191e5a4ac34b12b0bfc6237a2202"><img src="http://joss.theoj.org/papers/c73c191e5a4ac34b12b0bfc6237a2202/status.svg"></a>
Markdown: [![status](http://joss.theoj.org/papers/c73c191e5a4ac34b12b0bfc6237a2202/status.svg)](http://joss.theoj.org/papers/c73c191e5a4ac34b12b0bfc6237a2202)

Reviewer questions

Conflict of interest

• [x] As the reviewer I confirm that there are no conflicts of interest for me to review this work (such as being a major contributor to the software).

  General checks

• [x] Repository: Is the source code for this software available at the repository url?

• [x] License: Does the repository contain a plain-text LICENSE file with the contents of an OSI approved software license?

• [x] Version: Does the release version given match the GitHub release (v2.0.0)?

  Functionality

• [x] Installation: Does installation proceed as outlined in the documentation?

• [x] Functionality: Have the functional claims of the software been confirmed?

• [x] Performance: Have any performance claims of the software been confirmed?

  Documentation

• [x] A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?

• [x] Installation instructions: Is there a clearly-stated list of dependencies? Ideally these should be handled with an automated package management solution.
• [x] Example usage: Do the authors include examples of how to use the software (ideally to solve real-world analysis problems).
• [x] Functionality documentation: Is the core functionality of the software documented to a satisfactory level (e.g. API method documentation)?
• [x] Automated tests: Are there automated tests or manual steps described so that the function of the software can be verified?
• [x] Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support
  

  Software paper

Paper PDF: 10.21105.joss.00031.pdf

• [x] Authors: Does the paper.md file include a list of authors with their affiliations?
• [x] A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
• [x] References: Do all archival references that should have a DOI list one (e.g. papers, datasets, software)?

[arfon]

/ cc @openjournals/joss-reviewers - would anyone be willing to review this submission?

If you would like to review this submission then please comment on this thread so that others know you're doing a review (so as not to duplicate effort). Something as simple as :hand: I am reviewing this will suffice.

Reviewer instructions

• Please work through the checklist at the start of this issue.
• If you need any further guidance/clarification take a look at the reviewer guidelines here http://joss.theoj.org/about#reviewer_guidelines
• Please make a publication recommendation at the end of your review

Any questions, please ask for help by commenting on this issue! 🚀

[cMadan]

For the paper PDF, references after the first one aren't displayed quite right. Would using a semicolon between them (rather than a comma) be the fix?

[Kevin-Mattheus-Moerman]

:hand: I am reviewing this

[arfon]

For the paper PDF, references after the first one aren't displayed quite right. Would using a semicolon between them (rather than a comma) be the fix?

@cMadan - yep, I've fixed this in https://github.com/cMadan/prism/pull/1 and updated the PDF in the review issue.

[Kevin-Mattheus-Moerman]

I have reviewed this submission in terms of MATLAB implementation and installation. I have also verified the demo and benchmark work as described. However I am not an expert in the technical topic the software addresses. Would it be possible to provide a more graphical display of the functionality and technical claims of the software, i.e. plot graphs and figures visualizing the nature of the input data, the processing and the output results? Perhaps if the functionality can be described more thoroughly and graphically in this manor users new to this field may be able to grasp functionality better.

I tested this submission using MATLAB R2016b PreRelease.

I can confirm installation (of the software and the third party SparseBayes V2 toolbox) and implementation proceeded as described in the README.md.

I was able to install the software and run the demo.m.

Comments on paper:
Check spelling for “lregularization” in paper
Fix spelling error “which camn be obtained” in paper
Does the figure need a caption/description, like in the README.md?
Check the references. Is the author missing for the reference: “Sparse Bayesian Learning and the Relevance Vector Machine”

In README.md check spelling for: “regression lead to”. Should this be “leads”?

[labarba]

This opens a bigger question for me—as JOSS editor—, which is: Should we be publishing packages that rely on proprietary programming and execution environments?

A quick look at the README for this submission immediately struck me with: _"requires three first-party toolboxes: (1) Curve Fitting Toolbox; (2) Statistics and Machine Learning Toolbox; (3) Signal Processing Toolbox"_
Are these paid MATLAB Toolboxes?

Should we have a conversation about that on another Issue or in the Google Group?

[arfon]

This opens a bigger question for me—as JOSS editor—, which is: Should we be publishing packages that rely on proprietary programming and execution environments?

Yes I think these should be acceptable with the caveat that it should to be possible for a reviewer to address the following review points:

[ ] Installation: Does installation proceed as outlined in the documentation?
[ ] Functionality: Have the functional claims of the software been confirmed?
[ ] Performance: Have any performance claims of the software been confirmed?

Should we have a conversation about that on another Issue or in the Google Group?

Let's have this conversation in an issue. Are you happy to start it?

[cMadan]

@Kevin-Mattheus-Moerman, thanks for the review! I've fixed the typos and missing figure caption (was in paper.md, but not PDF). Reference is correct (is same as previous, 'line' notation is produced when PDF is compiled).
In demo.m, I have now added some scatter plots for evaluating the predictions.

@arfon, the subsequent references still look incorrect in the PDF. I've also made a few changes, so maybe you could re-compile the PDF?

@labarba, yes, these are paid MATLAB toolboxes, though (most) educational institutions that have MATLAB licences would also have these toolboxes. More broadly, I think any restriction on dependencies would preclude many MATLAB-based projects from being published in JOSS.

[arfon]

@cMadan - I've recompiled the paper here. Could you take a look and see if it's formatted better now?

[cMadan]

@arfon - much better! The only remaining issue is this "(e.g., (Sowell et al. 2003; Hogstrom et al. 2013; Madan and Kensinger 2016))". I'm not sure how I should change the md to make that turn out better. Effectively want something equivalent to "\cite

[nicoguaro]

I think that instead of the logo an image of a result would be much illustrative.

[arfon]

@cMadan - changing the Markdown to:

...quadratic regression are often used (e.g., @SoweEtal2003; @HogsEtal2013; @MadaKens2016), it has been shown that...

results in this output:

Does that look better? I'm not a pandoc expert I'm afraid.

[cMadan]

@nicoguaro - I've just added example output.

@arfon - that's sufficiently better, thanks! I really appreciate the help! I've updated paper.md on the repo.

[arfon]

@arfon - that's sufficiently better, thanks! I really appreciate the help! I've updated paper.md on the repo.

OK great. Here's the updated paper: 10.21105.joss.00031.pdf

[cMadan]

@arfon - that looks great, thanks!

[Kevin-Mattheus-Moerman]

Thanks for implementing the changes and for adding basic visualizations of results in demo.m. For the paper it seems that the reference with the title: "Sparse Bayesian Learning and the Relevance Vector Machine" misses an author field. Should this be Michael E. Tipping?
I've now ticked all the boxes above. Once the reference for the paper is verified I can recommend this paper for publication in JOSS.

[Kevin-Mattheus-Moerman]

There is as you know an ongoing discussion on the inclusion of software relying on proprietary programming environments. It may be that a final decision on your submission will be made after that discussion has been concluded. Apologies for the delay.

[nicoguaro]

Even though I'm not an editor I think that the output of the discussion should not influence the publication of this paper. This paper was submitted before the discussion even started, if it would have started before maybe the author would have not submitted.

[cMadan]

@Kevin-Mattheus-Moerman - The author is Tipping, but it is being displayed 'properly'. The PDF is compiled from markdown+bib by @arfon's code, and is based on an IEEE convention. See here: http://tex.stackexchange.com/questions/29381/is-it-normal-for-bibtex-to-replace-similar-author-names-with

[cMadan]

@arfon - I think all of the reviewer comments have been addressed, and the paper has been recommended for publication.

[arfon]

OK thanks @cMadan. I just need you to update this thread with the archive DOI (Zenodo/figshare etc) for the software before I can complete the submission.

[cMadan]

@arfon - the doi is 10.5281/zenodo.56821. I don't think I can edit the initial comment from @whedon.

[arfon]

OK thanks for the review here @Kevin-Mattheus-Moerman!

@cMadan - this is now accepted into JOSS, your DOI is: http://dx.doi.org/10.21105/joss.00031

🎉 🚀 💥

[cMadan]

I just noticed that under the second figure, it says "Figure 1" (that isn't in the markdown itself). I made a change to paper.md that hopefully fixes this.

[arfon]

@cMadan - could you take a look now and if this is better? https://github.com/openjournals/joss-papers/blob/master/joss.00031/10.21105.joss.00031.pdf

[cMadan]

@arfon - great, thank you!