Joss-reviews: [REVIEW]: WrightTools: a Python package for multidimensional spectroscopy

[whedon]

Submitting author: @untzag (Blaise Thompson)
Repository: https://github.com/wright-group/WrightTools
Version: 3.2.0
Editor: @yochannah
Reviewer: @daissi, @ivergara
Archive: 10.5281/zenodo.2541776

Status

Status badge code:

HTML: <a href="http://joss.theoj.org/papers/a82637112ac3e03df961d4494bc927d4"><img src="http://joss.theoj.org/papers/a82637112ac3e03df961d4494bc927d4/status.svg"></a>
Markdown: [![status](http://joss.theoj.org/papers/a82637112ac3e03df961d4494bc927d4/status.svg)](http://joss.theoj.org/papers/a82637112ac3e03df961d4494bc927d4)

Reviewers and authors:

Please avoid lengthy details of difficulties in the review thread. Instead, please create a new issue in the target repository and link to those issues (especially acceptance-blockers) in the review thread below. (For completists: if the target issue tracker is also on GitHub, linking the review thread in the issue or vice versa will create corresponding breadcrumb trails in the link target.)

Reviewer instructions & questions

@daissi & @ivergara, please carry out your review in this issue by updating the checklist below. If you cannot edit the checklist please:

1. Make sure you're logged in to your GitHub account
2. Be sure to accept the invite at this URL: https://github.com/openjournals/joss-reviews/invitations

The reviewer guidelines are available here: https://joss.theoj.org/about#reviewer_guidelines. Any questions/concerns please let @yochannah know.

✨ Please try and complete your review in the next two weeks ✨

Review checklist for @daissi

Conflict of interest

• [x] As the reviewer I confirm that I have read the JOSS conflict of interest policy and that there are no conflicts of interest for me to review this work.

Code of Conduct

• [x] I confirm that I read and will adhere to the JOSS code of conduct.

General checks

• [x] Repository: Is the source code for this software available at the repository url?
• [x] License: Does the repository contain a plain-text LICENSE file with the contents of an OSI approved software license?
• [x] Version: Does the release version given match the GitHub release (3.2.0)?
• [x] Authorship: Has the submitting author (@untzag) made major contributions to the software? Does the full list of paper authors seem appropriate and complete?

Functionality

• [x] Installation: Does installation proceed as outlined in the documentation?
• [x] Functionality: Have the functional claims of the software been confirmed?
• [x] Performance: If there are any performance claims of the software, have they been confirmed? (If there are no claims, please check off this item.)

Documentation

• [x] A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
• [x] Installation instructions: Is there a clearly-stated list of dependencies? Ideally these should be handled with an automated package management solution.
• [x] Example usage: Do the authors include examples of how to use the software (ideally to solve real-world analysis problems).
• [x] Functionality documentation: Is the core functionality of the software documented to a satisfactory level (e.g., API method documentation)?
• [x] Automated tests: Are there automated tests or manual steps described so that the function of the software can be verified?
• [x] Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support

Software paper

• [x] Authors: Does the paper.md file include a list of authors with their affiliations?
• [x] A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
• [x] References: Do all archival references that should have a DOI list one (e.g., papers, datasets, software)?

Review checklist for @ivergara

Conflict of interest

• [x] As the reviewer I confirm that I have read the JOSS conflict of interest policy and that there are no conflicts of interest for me to review this work.

Code of Conduct

• [x] I confirm that I read and will adhere to the JOSS code of conduct.

General checks

• [x] Repository: Is the source code for this software available at the repository url?
• [x] License: Does the repository contain a plain-text LICENSE file with the contents of an OSI approved software license?
• [x] Version: Does the release version given match the GitHub release (3.2.0)?
• [x] Authorship: Has the submitting author (@untzag) made major contributions to the software? Does the full list of paper authors seem appropriate and complete?

Functionality

• [x] Installation: Does installation proceed as outlined in the documentation?
• [x] Functionality: Have the functional claims of the software been confirmed?
• [x] Performance: If there are any performance claims of the software, have they been confirmed? (If there are no claims, please check off this item.)

Documentation

• [x] A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
• [x] Installation instructions: Is there a clearly-stated list of dependencies? Ideally these should be handled with an automated package management solution.
• [x] Example usage: Do the authors include examples of how to use the software (ideally to solve real-world analysis problems).
• [x] Functionality documentation: Is the core functionality of the software documented to a satisfactory level (e.g., API method documentation)?
• [x] Automated tests: Are there automated tests or manual steps described so that the function of the software can be verified?
• [x] Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support

Software paper

• [x] Authors: Does the paper.md file include a list of authors with their affiliations?
• [x] A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
• [x] References: Do all archival references that should have a DOI list one (e.g., papers, datasets, software)?

[whedon]

Hello human, I'm @whedon, a robot that can help you with some common editorial tasks. @daissi, it looks like you're currently assigned as the reviewer for this paper :tada:.

:star: Important :star:

If you haven't already, you should seriously consider unsubscribing from GitHub notifications for this (https://github.com/openjournals/joss-reviews) repository. As a reviewer, you're probably currently watching this repository which means for GitHub's default behaviour you will receive notifications (emails) for all reviews 😿

To fix this do the following two things:

1. Set yourself as 'Not watching' https://github.com/openjournals/joss-reviews:

1. You may also like to change your default settings for this watching repositories in your GitHub profile here: https://github.com/settings/notifications

For a list of things I can do to help you, just type:

@whedon commands

[whedon]

Attempting PDF compilation. Reticulating splines etc...

[whedon]

:point_right: Check article proof :page_facing_up: :point_left:

[daissi]

@untzag everything is fine, except some minor comments:

1. README.md and homepage (http://wright.tools/) don't provide a description of the purpose of WrightTools. It would be nice to have a small introduction in both of them like in the paper (e.g. extend the sentence "Tools for loading, processing, and plotting multidimensional spectroscopy data.").

2. There is no commit from the last author John C. Wright, could you clarify what is his role in WrightTools' development?

3. CONTRIBUTORS file contains list of contributors, but it could be useful to have a link in this file to the doc how to contribute: http://wright.tools/en/stable/contributing.html

4. In the paper, you said "To our knowledge, WrightTools is the first MDS-focused toolkit to be freely available and openly licensed" but there is a list of alternatives packages with similar goal as WrightTools: http://wright.tools/en/stable/alternatives.html So, what is the difference between WrightTools and the others?

5. In some example scripts (quickstart-3, 4 and 5) from http://wright.tools/en/stable/quickstart.html#visualize-data, there are missing 'plt.show()' call at the end (just to be consistent with quickstart-1 and 2).

[untzag]

@daissi thanks so much for your comments! I'll try to respond to them here, if that's appropriate.

1. README.md and homepage (http://wright.tools/) don't provide a description of the purpose of WrightTools. It would be nice to have a small introduction in both of them like in the paper (e.g. extend the sentence "Tools for loading, processing, and plotting multidimensional spectroscopy data.").

I agree. My coauthor @darienmorrow was kind enough to make a PR to adding such a description: https://github.com/wright-group/WrightTools/pull/852. It has been merged.

1. There is no commit from the last author John C. Wright, could you clarify what is his role in WrightTools' development?

The developers of WrightTools are mostly graduate students in John's research group. When submitting this paper, we read the JOSS authorship guidelines and talked among ourselves about whether John should be an author on this paper. While it's true that John has not contributed lines of code, he has involved himself significantly in ongoing conversations about how to best design software for MDS. He has been one of our most important users, and has given helpful feedback about what is confusing and what works well. Furthermore, we feel that WrightTools was shaped significantly by John's vision for high-throughput, flexible MDS instrumentation.

Speaking personally, I'm just not sure how to apply the authorship guidelines in this case. I would appreciate guidance from the reviewers.

1. CONTRIBUTORS file contains list of contributors, but it could be useful to have a link in this file to the doc how to contribute: http://wright.tools/en/stable/contributing.html

I agree. My coauthor @ksunden was kind enough to make a PR: https://github.com/wright-group/WrightTools/pull/853. It has been merged.

1. In the paper, you said "To our knowledge, WrightTools is the first MDS-focused toolkit to be freely available and openly licensed" but there is a list of alternatives packages with similar goal as WrightTools: http://wright.tools/en/stable/alternatives.html So, what is the difference between WrightTools and the others?

Simply put, WrightTools focuses on multidimensional spectroscopy. The packages listed under "alternatives" all have some overlap (multidimensional self-describing datasets, a focus on spectroscopy...) but none of them focus on MDS-specific processing challenges.

Perhaps the "alternatives" page could be amended to make this point clearer? I'll think on this more.

1. In some example scripts (quickstart-3, 4 and 5) from http://wright.tools/en/stable/quickstart.html#visualize-data, there are missing 'plt.show()' call at the end (just to be consistent with quickstart-1 and 2).

I agree. My coauthor @ksunden was kind enough to make a PR: https://github.com/wright-group/WrightTools/pull/850. It has been merged.

[daissi]

@daissi thanks so much for your comments! I'll try to respond to them here, if that's appropriate.

1. README.md and homepage (http://wright.tools/) don't provide a description of the purpose of WrightTools. It would be nice to have a small introduction in both of them like in the paper (e.g. extend the sentence "Tools for loading, processing, and plotting multidimensional spectroscopy data.").

I agree. My coauthor @darienmorrow was kind enough to make a PR to adding such a description: wright-group/WrightTools#852. It has been merged.

:thumbsup:

1. There is no commit from the last author John C. Wright, could you clarify what is his role in WrightTools' development?

The developers of WrightTools are mostly graduate students in John's research group. When submitting this paper, we read the JOSS authorship guidelines and talked among ourselves about whether John should be an author on this paper. While it's true that John has not contributed lines of code, he has involved himself significantly in ongoing conversations about how to best design software for MDS. He has been one of our most important users, and has given helpful feedback about what is confusing and what works well. Furthermore, we feel that WrightTools was shaped significantly by John's vision for high-throughput, flexible MDS instrumentation.

Speaking personally, I'm just not sure how to apply the authorship guidelines in this case. I would appreciate guidance from the reviewers.

In this case, it is fine to have John Wright as last author of the paper. :thumbsup:

1. CONTRIBUTORS file contains list of contributors, but it could be useful to have a link in this file to the doc how to contribute: http://wright.tools/en/stable/contributing.html

I agree. My coauthor @ksunden was kind enough to make a PR: wright-group/WrightTools#853. It has been merged.

:thumbsup:

1. In the paper, you said "To our knowledge, WrightTools is the first MDS-focused toolkit to be freely available and openly licensed" but there is a list of alternatives packages with similar goal as WrightTools: http://wright.tools/en/stable/alternatives.html So, what is the difference between WrightTools and the others?

Simply put, WrightTools focuses on multidimensional spectroscopy. The packages listed under "alternatives" all have some overlap (multidimensional self-describing datasets, a focus on spectroscopy...) but none of them focus on MDS-specific processing challenges.

Perhaps the "alternatives" page could be amended to make this point clearer? I'll think on this more.

You can keep the name "alternatives" for this page, but maybe you can add this description in the top of the page so that readers can have a direct preview of what they can expect from the other software.

1. In some example scripts (quickstart-3, 4 and 5) from http://wright.tools/en/stable/quickstart.html#visualize-data, there are missing 'plt.show()' call at the end (just to be consistent with quickstart-1 and 2).

I agree. My coauthor @ksunden was kind enough to make a PR: wright-group/WrightTools#850. It has been merged.

:thumbsup:

[ivergara]

@untzag I'd say that everything looks in order, just a few comments/questions beyond what the previous reviewer said

1. The tag hdf5 seems to me not relevant. hdf5 is more like a detail of implementation on how to store data for future use by the same tool than something relevant for a future user. For sure it's important and relevant in the context of implementation and future accessibility of the data.

2. You claim that

WrightTools defines a universal MDS data format

would you care to perhaps comment/expand on the universal character of it? I find the effort of trying to suggest a way to store the data in a more standardized way a good one. On the other hand creating a standard from small input doesn't seem universal to me. It's great that it's well documented and based on hdf5. Perhaps I'd suggest you to make an independent paper just discussing and defining this data format.

1. More of an observation, but the name WrightTools doesn't really evoke any kind of relationship with MDS or spectroscopy. Although it makes clear the origin of affiliation, but as such community contributions might suffer. I have the same kind of observation to the name of the data format.

[untzag]

@ivergara thanks for the review

1. The tag hdf5 seems to me not relevant. hdf5 is more like a detail of implementation on how to store data for future use by the same tool than something relevant for a future user. For sure it's important and relevant in the context of implementation and future accessibility of the data.

I don't have strong feelings about the tags, so I've made a PR to remove the hdf5 tag.

1. You claim that WrightTools defines a universal MDS data format would you care to perhaps comment/expand on the universal character of it? I find the effort of trying to suggest a way to store the data in a more standardized way a good one. On the other hand creating a standard from small input doesn't seem universal to me. It's great that it's well documented and based on hdf5. Perhaps I'd suggest you to make an independent paper just discussing and defining this data format.

Great question. I think your critique that we currently have a relatively small number of input formats is valid. When talking about "universality", what comes to my mind is the different kinds of multidimensional experiments that the wt5 file format can describe. I do think that our format is self-evidently extensible to a wide range of experimental combinations, and in fact we do regularly use WrightTools to process dramatically different multidimensional data-sets.

I love your idea of working to more formally describe our internal file conventions. That's something we're interested in, especially if it means that other members of our community can write interfaces to our format in their preferred language (e.g. MATLAB).

1. More of an observation, but the name WrightTools doesn't really evoke any kind of relationship with MDS or spectroscopy. Although it makes clear the origin of affiliation, but as such community contributions might suffer. I have the same kind of observation to the name of the data format.

Yeah, if we were starting it today I don't think we would name this package WrightTools. It was a great name when we first started out with a goal of sharing some code between graduate students in the Wright Group, but now that it has grown into something we want to share with everyone the name makes less sense... We'll think more carefully next time we name something :thinking:.

[yochannah]

Okay, it looks like all the checkboxes are checked - @untzag @daissi @ivergara - are we ready to publish do you think? 🎉

[untzag]

@yochannah everything looks good from my end

[daissi]

@yochannah that's fine with me.

[yochannah]

Fantastic! Okay, @untzag - At this point could you make an archive of the reviewed software in Zenodo/figshare/other service and update this thread with the DOI of the archive?

[untzag]

@yochannah hope this is helpful!

doi for all versions: 10.5281/zenodo.1198904
doi for reviewed version: 10.5281/zenodo.2541196

thanks so much

[yochannah]

@whedon generate pdf

[whedon]

Attempting PDF compilation. Reticulating splines etc...

[whedon]

:point_right: Check article proof :page_facing_up: :point_left:

[yochannah]

@untzag Thanks for this! I have a couple of minor hopefully final comments -

First - The list of paper authors in the proof above is slightly different from the list shown in the archive. We should either add them all to the paper or remove from the archive by editing the metadata.

Second, I noticed while I was doing a final scan of the manuscript that the full stop / period often comes before the citation which is a bit odd -

Matplotlib supports one, two, and even three-dimensional plotting. (Hunter, 2007)

It would be a little bit tidier to make sure the citation is in the same sentence, i.e.

Matplotlib supports one, two, and even three-dimensional plotting (Hunter, 2007).

This happens several times throughout the paper - would you mind updating all instances?

Once this is done, I think we'll be ready to go!

[untzag]

@yochannah

The list of paper authors in the proof above is slightly different from the list shown in the archive. We should either add them all to the paper or remove from the archive by editing the metadata.

In this case I think the best thing to do is to create a one-off zenodo entry for JOSS:

10.5281/zenodo.2541776

Hope this works for you.

It would be a little bit tidier to make sure the citation is in the same sentence.

Done and merged.

[yochannah]

@whedon generate pdf

[whedon]

Attempting PDF compilation. Reticulating splines etc...

[whedon]

:point_right: Check article proof :page_facing_up: :point_left:

[yochannah]

@whedon set 10.5281/zenodo.2541776 as archive

[whedon]

OK. 10.5281/zenodo.2541776 is the archive.

[yochannah]

@arfon Everything looks in order here - I think we're ready to accept and publish! 📖

Thanks @untzag for submitting this paper and working with us to get this ready for publication, and thank you @ivergara and @daissi for putting in time and effort to review. 💯 🥇

[arfon]

@whedon accept

[whedon]

Attempting dry run of processing paper acceptance...

[whedon]

Check final proof :point_right: https://github.com/openjournals/joss-papers/pull/439

If the paper PDF and Crossref deposit XML look good in https://github.com/openjournals/joss-papers/pull/439, then you can now move forward with accepting the submission by compiling again with the flag deposit=true e.g.
@whedon accept deposit=true

[arfon]

@whedon accept deposit=true

[whedon]

Doing it live! Attempting automated processing of paper acceptance...

[whedon]

🚨🚨🚨 THIS IS NOT A DRILL, YOU HAVE JUST ACCEPTED A PAPER INTO JOSS! 🚨🚨🚨

Here's what you must now do:

1. Check final PDF and Crossref metadata that was deposited :point_right: https://github.com/openjournals/joss-papers/pull/440
2. Wait a couple of minutes to verify that the paper DOI resolves https://doi.org/10.21105/joss.01141
3. If everything looks good, then close this review issue.

4. Party like you just published a paper! 🎉🌈🦄💃👻🤘

   Any issues? notify your editorial technical team...

[arfon]

@daissi, @ivergara - many thanks for your reviews here and to @yochannah for editing this submission ✨

@untzag - your paper is now accepted into JOSS :zap::rocket::boom:

[whedon]

:tada::tada::tada: Congratulations on your paper acceptance! :tada::tada::tada:

If you would like to include a link to your paper from your README use the following code snippets:

Markdown:
[![DOI](http://joss.theoj.org/papers/10.21105/joss.01141/status.svg)](https://doi.org/10.21105/joss.01141)

HTML:
<a style="border-width:0" href="https://doi.org/10.21105/joss.01141">
  <img src="http://joss.theoj.org/papers/10.21105/joss.01141/status.svg" alt="DOI badge" >
</a>

reStructuredText:
.. image:: http://joss.theoj.org/papers/10.21105/joss.01141/status.svg
   :target: https://doi.org/10.21105/joss.01141

This is how it will look in your documentation:

We need your help!

Journal of Open Source Software is a community-run journal and relies upon volunteer effort. If you'd like to support us please consider doing either one (or both) of the the following:

• Volunteering to review for us sometime in the future. You can add your name to the reviewer list here: http://joss.theoj.org/reviewer-signup.html
• Making a small donation to support our running costs here: https://www.flipcause.com/secure/cause_pdetails/Mjk3Nzk=