Joss-reviews: [REVIEW]: Viime: Visualization and Integration of Metabolomics Experiments

[whedon]

Submitting author: @waxlamp (Roni Choudhury)
Repository: https://github.com/girder/viime
Version: v1.0.1
Editor: @lpantano
Reviewers: @joannawolthuis, @rowlandm, @jkanche
Archive: 10.5281/zenodo.4086165

:warning: JOSS reduced service mode :warning:

Due to the challenges of the COVID-19 pandemic, JOSS is currently operating in a "reduced service mode". You can read more about what that means in our blog post.

Status

Status badge code:

HTML: <a href="https://joss.theoj.org/papers/77cf7a1d8ffd00c8278a42c449914bb1"><img src="https://joss.theoj.org/papers/77cf7a1d8ffd00c8278a42c449914bb1/status.svg"></a>
Markdown: [![status](https://joss.theoj.org/papers/77cf7a1d8ffd00c8278a42c449914bb1/status.svg)](https://joss.theoj.org/papers/77cf7a1d8ffd00c8278a42c449914bb1)

Reviewers and authors:

Please avoid lengthy details of difficulties in the review thread. Instead, please create a new issue in the target repository and link to those issues (especially acceptance-blockers) by leaving comments in the review thread below. (For completists: if the target issue tracker is also on GitHub, linking the review thread in the issue or vice versa will create corresponding breadcrumb trails in the link target.)

Reviewer instructions & questions

@joannawolthuis & @rowlandm, please carry out your review in this issue by updating the checklist below. If you cannot edit the checklist please:

1. Make sure you're logged in to your GitHub account
2. Be sure to accept the invite at this URL: https://github.com/openjournals/joss-reviews/invitations

The reviewer guidelines are available here: https://joss.readthedocs.io/en/latest/reviewer_guidelines.html. Any questions/concerns please let @lpantano know.

✨ Please try and complete your review in the next six weeks ✨

Review checklist for @joannawolthuis

Conflict of interest

• [x] I confirm that I have read the JOSS conflict of interest (COI) policy and that: I have no COIs with reviewing this work or that any perceived COIs have been waived by JOSS for the purpose of this review.

Code of Conduct

• [x] I confirm that I read and will adhere to the JOSS code of conduct.

General checks

• [x] Repository: Is the source code for this software available at the repository url?
• [x] License: Does the repository contain a plain-text LICENSE file with the contents of an OSI approved software license?
• [x] Contribution and authorship: Has the submitting author (@waxlamp) made major contributions to the software? Does the full list of paper authors seem appropriate and complete?

Functionality

• [x] Installation: Does installation proceed as outlined in the documentation?
• [x] Functionality: Have the functional claims of the software been confirmed?
• [x] Performance: If there are any performance claims of the software, have they been confirmed? (If there are no claims, please check off this item.)

Documentation

• [x] A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
• [x] Installation instructions: Is there a clearly-stated list of dependencies? Ideally these should be handled with an automated package management solution.
• [x] Example usage: Do the authors include examples of how to use the software (ideally to solve real-world analysis problems).
• [x] Functionality documentation: Is the core functionality of the software documented to a satisfactory level (e.g., API method documentation)?
• [x] Automated tests: Are there automated tests or manual steps described so that the functionality of the software can be verified?
• [x] Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support

Software paper

• [x] Summary: Has a clear description of the high-level functionality and purpose of the software for a diverse, non-specialist audience been provided?
• [x] A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
• [x] State of the field: Do the authors describe how this software compares to other commonly-used packages?
• [x] Quality of writing: Is the paper well written (i.e., it does not require editing for structure, language, or writing quality)?
• [x] References: Is the list of references complete, and is everything cited appropriately that should be cited (e.g., papers, datasets, software)? Do references in the text use the proper citation syntax?

Review checklist for @jkanche

Conflict of interest

• [x] I confirm that I have read the JOSS conflict of interest (COI) policy and that: I have no COIs with reviewing this work or that any perceived COIs have been waived by JOSS for the purpose of this review.

Code of Conduct

• [x] I confirm that I read and will adhere to the JOSS code of conduct.

General checks

• [x] Repository: Is the source code for this software available at the repository url?
• [x] License: Does the repository contain a plain-text LICENSE file with the contents of an OSI approved software license?
• [x] Contribution and authorship: Has the submitting author (@waxlamp) made major contributions to the software? Does the full list of paper authors seem appropriate and complete?

Functionality

• [x] Installation: Does installation proceed as outlined in the documentation?
• [x] Functionality: Have the functional claims of the software been confirmed?
• [x] Performance: If there are any performance claims of the software, have they been confirmed? (If there are no claims, please check off this item.)

Documentation

• [x] A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
• [x] Installation instructions: Is there a clearly-stated list of dependencies? Ideally these should be handled with an automated package management solution.
• [x] Example usage: Do the authors include examples of how to use the software (ideally to solve real-world analysis problems).
• [x] Functionality documentation: Is the core functionality of the software documented to a satisfactory level (e.g., API method documentation)?
• [x] Automated tests: Are there automated tests or manual steps described so that the functionality of the software can be verified?
• [x] Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support

Software paper

• [x] Summary: Has a clear description of the high-level functionality and purpose of the software for a diverse, non-specialist audience been provided?
• [x] A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
• [x] State of the field: Do the authors describe how this software compares to other commonly-used packages?
• [x] Quality of writing: Is the paper well written (i.e., it does not require editing for structure, language, or writing quality)?
• [x] References: Is the list of references complete, and is everything cited appropriately that should be cited (e.g., papers, datasets, software)? Do references in the text use the proper citation syntax?

[whedon]

Hello human, I'm @whedon, a robot that can help you with some common editorial tasks. @joannawolthuis, @rowlandm it looks like you're currently assigned to review this paper :tada:.

:warning: JOSS reduced service mode :warning:

Due to the challenges of the COVID-19 pandemic, JOSS is currently operating in a "reduced service mode". You can read more about what that means in our blog post.

:star: Important :star:

If you haven't already, you should seriously consider unsubscribing from GitHub notifications for this (https://github.com/openjournals/joss-reviews) repository. As a reviewer, you're probably currently watching this repository which means for GitHub's default behaviour you will receive notifications (emails) for all reviews 😿

To fix this do the following two things:

1. Set yourself as 'Not watching' https://github.com/openjournals/joss-reviews:

1. You may also like to change your default settings for this watching repositories in your GitHub profile here: https://github.com/settings/notifications

For a list of things I can do to help you, just type:

@whedon commands

For example, to regenerate the paper pdf after making changes in the paper's md or bib files, type:

@whedon generate pdf

[whedon]

PDF failed to compile for issue #2410 with the following error:

Can't find any papers to compile :-(

[lpantano]

@whedon generate pdf from branch joss-paper

[whedon]

Attempting PDF compilation from custom branch joss-paper. Reticulating splines etc...

[whedon]

:point_right: Check article proof :page_facing_up: :point_left:

[lpantano]

@whedon check references

[lpantano]

@luizirber, would you like to review this tool?

[lpantano]

@waxlamp, if you know somebody who could be a reviewer let me know, still trying to find the second one. Thanks

[lpantano]

@camillescott, would you like to review this visualization tool of metabolomic data?

[lpantano]

Hi @a-slide, @rabdill, Would you like to review this tool? thanks!

[rabdill]

@lpantano this looks like a really cool project, but I just don't have time to do a review right now, I'm sorry.

[lpantano]

Hi @jkanche, @carleshf, would you like to review this tool? thanks!

[jkanche]

I am happy to review this!

[lpantano]

@whedon add @jkanche as reviewer

[whedon]

OK, @jkanche is now a reviewer

[joannawolthuis]

Hi @lpantano ! I'm taking a look - but here's a perhaps noob question: this is my first review, should I be writing a more extensive document (and if so, where should I submit it?) using the checklist as a backbone/guide? Or is this a very modern type of review where it is just the checklist? :)

[lpantano]

Hi @joannawolthuis, you don't need to write an extensive review. If you want to comment on something, you can do it. But the important it all the items in the checklist are checked. If not, you can open an issue in the tool repository to address the issue, so they can fix it and you can mark all the items in the checklist.

[waxlamp]

@waxlamp, if you know somebody who could be a reviewer let me know, still trying to find the second one. Thanks

Hi @lpantano! Looks like we have two reviewers now, is that right? I have a suggested name in case you need someone else.

[lpantano]

yes, we have two!

[waxlamp]

Great, thanks for confirming!

[carleshf]

Hi!

Sorry for the delay, I'm happy to review the tool too if I'm not too late.

Carles Hernández

Missatge de Roni Choudhury [e-mail] del dia dl., 27 de
jul. 2020 a les 23:59:

Great, thanks for confirming!

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/openjournals/joss-reviews/issues/2410#issuecomment-664659624,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAPTRZALVTBISIX3XPXFCELR5X2DVANCNFSM4OMEBJZA
.

[lpantano]

Hey @joannawolthuis, @jkanche, how is the reviewing going? Any updates on this?

Thanks @carleshf, hopefully we don't need more reviewers for now. Thank you!

[jkanche]

Hi @lpantano , I opened an issue on the Viime repo, once they are figured/answered, I can get back to the review

[joannawolthuis]

Hi @lpantano! Same as @jkanche, have opened some issues. One question - for the 'installation' checkmark, does that still count if this aimed to be a webserver? If so I need to ask them to clarify their installation procedure..

[jkanche]

I agree with @joannawolthuis, I would also like to see more detailed installation instructions. I am confused on how one can setup an instance Viime for their datasets.

May be provide a docker container or instructions on how one can set everything up will be good.

[lpantano]

Thank you for the updates @joannawolthuis, @jkanche. Yes, the installation needs to be addressed as well for servers. It is a very good point.

[joannawolthuis]

Just checking in, thanks for your patience! I have opened some more issues. The installation works well now, just have some issues on startup in terms of example datasets etc. (opened an issue on their github)
Just another question - they are currently dealing with some bugs, as one always has with a tool, new updates will always add new bugs - should i only check off the mark for expected/promised functionality once they have ironed those out? (some plots are just not showing at all, etc).

[joannawolthuis]

Oh, one more - is there a supplemental info or something for this paper? Otherwise they have no comparison at all with other tools/state of the art, should I address that on their Github page or through another channel?

[joannawolthuis]

@lpantano, whoops forgot to tag you :)

[lpantano]

@joannawolthuis, if they don't point to anything else in the paper, there is not. You can address this by an issue in their repo and you can add that is related to this review. Thanks!

[joannawolthuis]

@lpantano Alright, thanks, opened an issue! And for their bugfixing, I'll just wait until they get it all working properly and then check the box, is that OK? :)

[lpantano]

Absolutely!

[waxlamp]

@lpantano, @joannawolthuis, and @jkanche, thank you all for serving as reviewers and providing valuable feedback to us!

I've been trying to keep up with the various issues and requests, and I believe I've at least addressed everything that has been brought up so far. That is, we have fixed some bugs, made documentation updates to explain certain things better, and filed issues for other bugs. My goal is to get the software and paper up to review standards, since (unfortunately) we don't have the person-hours to fix everything, while filing issues for the rest (to be fixed once we have more funding/FTEs).

I am hoping to sync up with everyone by sending this message. When you get a chance, could you look things over and let me know what still needs to be done?

Thanks again for your time and effort in being our reviewers!

[lpantano]

Hi @joannawolthuis , @jkanche, let me know if the last version addresses all your comments. Thanks!

[joannawolthuis]

@whedon generate pdf from branch joss-paper

[whedon]

Attempting PDF compilation from custom branch joss-paper. Reticulating splines etc...

[whedon]

:point_right: Check article proof :page_facing_up: :point_left:

[joannawolthuis]

@lpantano Hi! There are still some bugs open, but I know from experience that that can keep going on forever, so I will not hunt for more after the ones I currently addressed are fixed, then that is done :) I would ideally like some improvements to the State of the Field section, I opened an issue here, do you agree @jkanche? https://github.com/girder/viime/issues/624

[lpantano]

Thank you @joannawolthuis.
@jkanche, can you share your thoughts for the state of the changes? Thanks

[jkanche]

Most of the issues I had before were addressed. I agree with @joannawolthuis on the "Related Work" section. The focus of this section has been based on one tool - MetaboAnalyst. I think it would be great if the authors can talk about other metabolomics tools out there.

[waxlamp]

Thank you for the feedback. My team is working on improving the related work section, and I will send a ping from here when we're ready for a new round of reviewing.

[waxlamp]

@whedon generate pdf from branch joss-paper

[whedon]

Attempting PDF compilation from custom branch joss-paper. Reticulating splines etc...

[whedon]

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

[waxlamp]

@whedon generate pdf from branch joss-paper

[whedon]

Attempting PDF compilation from custom branch joss-paper. Reticulating splines etc...

[whedon]

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

[waxlamp]

@lpantano, @joannawolthuis, @jkanche: we have reworked the Related Work section to discuss a wider range of tools in the space. I think it does a good job of surveying the field and explaining what makes Viime unique.

[lpantano]

@joannawolthuis, @jkanche, let me know if this addresses everything. Thanks!

[jkanche]

@lpantano I think this addresses most of my concerns! Thanks for the updates @waxlamp!

[joannawolthuis]

@lpantano Same here, looks good! :) Thanks @waxlamp and VIIME team!

[lpantano]

Excellent! thank you so much.

[lpantano]

@waxlamp, can you create a zenodo archive of the repository and share here? Title and authors should match the paper information. Thanks!

[waxlamp]

@waxlamp, can you create a zenodo archive of the repository and share here? Title and authors should match the paper information. Thanks!

Definitely. The team is working on fixing some typos (no changes to content) and once that's ready, I will take care of this. Thank you!

[waxlamp]

@waxlamp, can you create a zenodo archive of the repository and share here? Title and authors should match the paper information. Thanks!

@lpantano, here you go: https://zenodo.org/record/4075591, with DOI 10.5281/zenodo.4075591.

Please let me know if there's anything else I can do.

[lpantano]

Thanks, this should work. I just noticed that the version in setup.py is 0.1.0: https://github.com/girder/viime/blob/master/setup.py#L26. Any chance to update that to the right number and update the tar.gz file so the 1.0.0 version match the code version? it is ok if you need to create a 1.0.1 or any other solution. Thanks!

[waxlamp]

Thank you for catching this, @lpantano. I just reuploaded a fixed version to Zenodo: https://zenodo.org/record/4086165, with DOI 10.5281/zenodo.4086165.

Hopefully this is ok for proceeding. If you think it makes sense, I can also delete the 1.0.0 bundle from Zenodo; please advise.

Thanks!

[lpantano]

@whedon set 10.5281/zenodo.4086165 as archive

[whedon]

OK. 10.5281/zenodo.4086165 is the archive.

[lpantano]

@whedon set v1.0.1 as version

[whedon]

OK. v1.0.1 is the version.

[lpantano]

@whedon accept

[whedon]

Attempting dry run of processing paper acceptance...

[whedon]

Reference check summary (note 'MISSING' DOIs are suggestions that need verification):

OK DOIs

- 10.1007/s11306-016-1094-6 is OK
- 10.1038/nrd.2016.32 is OK
- 10.1186/1471-2164-7-142 is OK
- 10.1007/s00216-016-9538-4 is OK
- 10.1002/wics.1246 is OK
- 10.1093/nar/gkp356 is OK
- 10.1093/nar/gky310 is OK
- 10.1002/elps.201500352 is OK
- 10.1007/s11306-017-1242-7 is OK
- 10.1186/1471-2105-11-395 is OK
- 10.1093/bioinformatics/btu136 is OK
- 10.1021/ac1021166 is OK
- 10.1002/0471250953.bi1411s37 is OK
- 10.1093/bioinformatics/btu813 is OK
- 10.1186/s13742-016-0115-8 is OK
- 10.1007/978-1-0716-0239-3_17 is OK
- 10.1021/ac300698c is OK

MISSING DOIs

- None

INVALID DOIs

- None

[whedon]

:wave: @openjournals/joss-eics, this paper is ready to be accepted and published.

Check final proof :point_right: https://github.com/openjournals/joss-papers/pull/1819

If the paper PDF and Crossref deposit XML look good in https://github.com/openjournals/joss-papers/pull/1819, then you can now move forward with accepting the submission by compiling again with the flag deposit=true e.g.
@whedon accept deposit=true

[danielskatz]

👋 @waxlamp - There are some small changes needed in the paper. See https://github.com/girder/viime/pull/687. Also, in line 194 of that file, is "-log10" correct? I don't understand the "-".

Finally, for @lpantano - this paper is far, far too long. Much of the content should be in the documentation, and just pointed to from the paper. The ideal for a JOSS paper is 250-1000 words, as described in https://joss.readthedocs.io/en/latest/submitting.html#what-should-my-paper-contain At this point, we'll let this proceed without asking the authors to cut it, but for next time, please check the rough length of the paper during the review process.

[lpantano]

Thanks, I will do that. I wasnt sure how much longer was the limit(sorry).
The -log10 of the p-value is a common transformation to get small values
(<0.00) to be large positive numbers. I think probably the transformation
is -1log10.

[waxlamp]

@waxlamp - There are some small changes needed in the paper. See girder/viime#687. Also, in line 194 of that file, is "-log10" correct? I don't understand the "-".

Thank you!

As @lpantano mentioned, "-log10" refers to negative log-base-10 of the p-value.

@lpantano, now that the requested fixes are applied, what are the next steps?

[danielskatz]

that's it

[danielskatz]

@whedon accept

[whedon]

Attempting dry run of processing paper acceptance...

[whedon]

Reference check summary (note 'MISSING' DOIs are suggestions that need verification):

OK DOIs

- 10.1007/s11306-016-1094-6 is OK
- 10.1038/nrd.2016.32 is OK
- 10.1186/1471-2164-7-142 is OK
- 10.1007/s00216-016-9538-4 is OK
- 10.1002/wics.1246 is OK
- 10.1093/nar/gkp356 is OK
- 10.1093/nar/gky310 is OK
- 10.1002/elps.201500352 is OK
- 10.1007/s11306-017-1242-7 is OK
- 10.1186/1471-2105-11-395 is OK
- 10.1093/bioinformatics/btu136 is OK
- 10.1021/ac1021166 is OK
- 10.1002/0471250953.bi1411s37 is OK
- 10.1093/bioinformatics/btu813 is OK
- 10.1186/s13742-016-0115-8 is OK
- 10.1007/978-1-0716-0239-3_17 is OK
- 10.1021/ac300698c is OK

MISSING DOIs

- None

INVALID DOIs

- None

[whedon]

:wave: @openjournals/joss-eics, this paper is ready to be accepted and published.

Check final proof :point_right: https://github.com/openjournals/joss-papers/pull/1822

If the paper PDF and Crossref deposit XML look good in https://github.com/openjournals/joss-papers/pull/1822, then you can now move forward with accepting the submission by compiling again with the flag deposit=true e.g.
@whedon accept deposit=true

[danielskatz]

@whedon accept deposit=true

[whedon]

Doing it live! Attempting automated processing of paper acceptance...

[whedon]

🐦🐦🐦 👉 Tweet for this paper 👈 🐦🐦🐦

[whedon]

🚨🚨🚨 THIS IS NOT A DRILL, YOU HAVE JUST ACCEPTED A PAPER INTO JOSS! 🚨🚨🚨

Here's what you must now do:

1. Check final PDF and Crossref metadata that was deposited :point_right: https://github.com/openjournals/joss-papers/pull/1823
2. Wait a couple of minutes to verify that the paper DOI resolves https://doi.org/10.21105/joss.02410
3. If everything looks good, then close this review issue.

4. Party like you just published a paper! 🎉🌈🦄💃👻🤘

   Any issues? Notify your editorial technical team...

[danielskatz]

Congratulations to @waxlamp (Roni Choudhury) and co-authors!!

And thanks to @lpantano for editing, and @joannawolthuis & @jkanche for reviewing!

[whedon]

:tada::tada::tada: Congratulations on your paper acceptance! :tada::tada::tada:

If you would like to include a link to your paper from your README use the following code snippets:

Markdown:
[![DOI](https://joss.theoj.org/papers/10.21105/joss.02410/status.svg)](https://doi.org/10.21105/joss.02410)

HTML:
<a style="border-width:0" href="https://doi.org/10.21105/joss.02410">
  <img src="https://joss.theoj.org/papers/10.21105/joss.02410/status.svg" alt="DOI badge" >
</a>

reStructuredText:
.. image:: https://joss.theoj.org/papers/10.21105/joss.02410/status.svg
   :target: https://doi.org/10.21105/joss.02410

This is how it will look in your documentation:

We need your help!

Journal of Open Source Software is a community-run journal and relies upon volunteer effort. If you'd like to support us please consider doing either one (or both) of the the following:

• Volunteering to review for us sometime in the future. You can add your name to the reviewer list here: https://joss.theoj.org/reviewer-signup.html
• Making a small donation to support our running costs here: https://numfocus.org/donate-to-joss

[waxlamp]

Thank you @lpantano, @joannawolthuis, @jkanche, and @danielskatz!