Joss-reviews: [REVIEW]: tidyHeatmap: an R package for modular heatmap production based on tidy principles

[whedon]

Submitting author: @stemangiola (Stefano Mangiola)
Repository: https://github.com/stemangiola/tidyHeatmap
Version: v1.1.2
Editor: @karthik
Reviewer: @rlbarter, @AliciaSchep
Archive: 10.5281/zenodo.3966515

:warning: JOSS reduced service mode :warning:

Due to the challenges of the COVID-19 pandemic, JOSS is currently operating in a "reduced service mode". You can read more about what that means in our blog post.

Status

Status badge code:

HTML: <a href="https://joss.theoj.org/papers/e07d07afe32b22aa787e7f01751c1644"><img src="https://joss.theoj.org/papers/e07d07afe32b22aa787e7f01751c1644/status.svg"></a>
Markdown: [![status](https://joss.theoj.org/papers/e07d07afe32b22aa787e7f01751c1644/status.svg)](https://joss.theoj.org/papers/e07d07afe32b22aa787e7f01751c1644)

Reviewers and authors:

Please avoid lengthy details of difficulties in the review thread. Instead, please create a new issue in the target repository and link to those issues (especially acceptance-blockers) by leaving comments in the review thread below. (For completists: if the target issue tracker is also on GitHub, linking the review thread in the issue or vice versa will create corresponding breadcrumb trails in the link target.)

Reviewer instructions & questions

@rlbarter & @AliciaSchep, please carry out your review in this issue by updating the checklist below. If you cannot edit the checklist please:

1. Make sure you're logged in to your GitHub account
2. Be sure to accept the invite at this URL: https://github.com/openjournals/joss-reviews/invitations

The reviewer guidelines are available here: https://joss.readthedocs.io/en/latest/reviewer_guidelines.html. Any questions/concerns please let @karthik know.

✨ Please try and complete your review in the next six weeks ✨

Review checklist for @rlbarter

Conflict of interest

• [x] I confirm that I have read the JOSS conflict of interest (COI) policy and that: I have no COIs with reviewing this work or that any perceived COIs have been waived by JOSS for the purpose of this review.

Code of Conduct

• [x] I confirm that I read and will adhere to the JOSS code of conduct.

General checks

• [x] Repository: Is the source code for this software available at the repository url?
• [x] License: Does the repository contain a plain-text LICENSE file with the contents of an OSI approved software license?
• [x] Contribution and authorship: Has the submitting author (@stemangiola) made major contributions to the software? Does the full list of paper authors seem appropriate and complete?
• [x] Substantial scholarly effort: Does this submission meet the scope eligibility described in the JOSS guidelines

Functionality

• [x] Installation: Does installation proceed as outlined in the documentation?
• [x] Functionality: Have the functional claims of the software been confirmed?
• [x] Performance: If there are any performance claims of the software, have they been confirmed? (If there are no claims, please check off this item.)

Documentation

• [x] A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
• [x] Installation instructions: Is there a clearly-stated list of dependencies? Ideally these should be handled with an automated package management solution.
• [x] Example usage: Do the authors include examples of how to use the software (ideally to solve real-world analysis problems).
• [x] Functionality documentation: Is the core functionality of the software documented to a satisfactory level (e.g., API method documentation)?
• [x] Automated tests: Are there automated tests or manual steps described so that the functionality of the software can be verified?
• [x] Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support

Software paper

• [x] Summary: Has a clear description of the high-level functionality and purpose of the software for a diverse, non-specialist audience been provided?
• [x] A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
• [x] State of the field: Do the authors describe how this software compares to other commonly-used packages?
• [x] Quality of writing: Is the paper well written (i.e., it does not require editing for structure, language, or writing quality)?
• [x] References: Is the list of references complete, and is everything cited appropriately that should be cited (e.g., papers, datasets, software)? Do references in the text use the proper citation syntax?

Review checklist for @AliciaSchep

Conflict of interest

• [x] I confirm that I have read the JOSS conflict of interest (COI) policy and that: I have no COIs with reviewing this work or that any perceived COIs have been waived by JOSS for the purpose of this review.

Code of Conduct

• [x] I confirm that I read and will adhere to the JOSS code of conduct.

General checks

• [x] Repository: Is the source code for this software available at the repository url?
• [x] License: Does the repository contain a plain-text LICENSE file with the contents of an OSI approved software license?
• [x] Contribution and authorship: Has the submitting author (@stemangiola) made major contributions to the software? Does the full list of paper authors seem appropriate and complete?
• [x] Substantial scholarly effort: Does this submission meet the scope eligibility described in the JOSS guidelines

Functionality

• [x] Installation: Does installation proceed as outlined in the documentation?
• [x] Functionality: Have the functional claims of the software been confirmed?
• [x] Performance: If there are any performance claims of the software, have they been confirmed? (If there are no claims, please check off this item.)

Documentation

• [x] A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
• [x] Installation instructions: Is there a clearly-stated list of dependencies? Ideally these should be handled with an automated package management solution.
• [x] Example usage: Do the authors include examples of how to use the software (ideally to solve real-world analysis problems).
• [x] Functionality documentation: Is the core functionality of the software documented to a satisfactory level (e.g., API method documentation)?
• [x] Automated tests: Are there automated tests or manual steps described so that the functionality of the software can be verified?
• [x] Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support

Software paper

• [x] Summary: Has a clear description of the high-level functionality and purpose of the software for a diverse, non-specialist audience been provided?
• [x] A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
• [x] State of the field: Do the authors describe how this software compares to other commonly-used packages?
• [x] Quality of writing: Is the paper well written (i.e., it does not require editing for structure, language, or writing quality)?
• [x] References: Is the list of references complete, and is everything cited appropriately that should be cited (e.g., papers, datasets, software)? Do references in the text use the proper citation syntax?

[whedon]

Hello human, I'm @whedon, a robot that can help you with some common editorial tasks. @rlbarter, @AliciaSchep it looks like you're currently assigned to review this paper :tada:.

:warning: JOSS reduced service mode :warning:

Due to the challenges of the COVID-19 pandemic, JOSS is currently operating in a "reduced service mode". You can read more about what that means in our blog post.

:star: Important :star:

If you haven't already, you should seriously consider unsubscribing from GitHub notifications for this (https://github.com/openjournals/joss-reviews) repository. As a reviewer, you're probably currently watching this repository which means for GitHub's default behaviour you will receive notifications (emails) for all reviews 😿

To fix this do the following two things:

1. Set yourself as 'Not watching' https://github.com/openjournals/joss-reviews:

1. You may also like to change your default settings for this watching repositories in your GitHub profile here: https://github.com/settings/notifications

For a list of things I can do to help you, just type:

@whedon commands

For example, to regenerate the paper pdf after making changes in the paper's md or bib files, type:

@whedon generate pdf

[whedon]

Reference check summary:

OK DOIs

- 10.21105/joss.01686 is OK
- 10.1093/bioinformatics/btw313 is OK
- 10.1093/bioinformatics/btu393 is OK

MISSING DOIs

- None

INVALID DOIs

- None

[whedon]

:point_right: Check article proof :page_facing_up: :point_left:

[rlbarter]

Created an issue in target repository due to installation issues: https://github.com/stemangiola/tidyHeatmap/issues/38

[rlbarter]

Issue resolved - was my laptop acting funky due to crashing earlier. A restart solved the problem.

[rlbarter]

I've completed my review.

This looks great overall.

I have a few very minor comments for the authors, which they can feel free to take or disregard.

• I noticed a minor typo in contribution section of README: "pleae open an issue" --> "please open an issue". Maybe just do a full spell-check?

• In the README, consider updating gather to pivot_longer just to be up-to-date with tidyverse function.

• In the README, use -> instead of =, just to follow usual convention.

• My heatmap package superheat was one of the early packages to add point, bar, line annotation to heatmaps, so it would be a relevant citation (https://www.tandfonline.com/doi/abs/10.1080/10618600.2018.1473780). Your package works much better with the modern tidyverse style (mine was built way back in 2015) - nicely done.

[stemangiola]

Thanks a lot @rlbarter all four suggestions have been incorporated in this pull request. Your package has great graphics, probably the cleanest of all.

[AliciaSchep]

Great work -- really nice interface for making heatmaps from dataframes!

In terms of the criteria for review I saw two main areas for potential improvement --

(1) The tests appear to mostly check that the functions run without error and create an output of the right class. This strategy seems like it could miss many potential issues. One package that can be helpful for visual regression testing is vdiffr. Or perhaps since this package is relying on ComplexHeatmap, testing for the actual rendered image is a step too far, but there could be tests to make sure that InputHeatmap objects get correctly converted into the right objects from ComplexHeatmap.

(2) I think both the paper and the documentation could benefit from a bit more explanation of how the various parts of a complex heatmap map to the "tidy" data framework. For example in the mtcars example, why do the hp and vs variables not get pulled into the Property column? My first instinct for tidying that data frame would be that all variables get pulled into Property. It seems like the difference with those in terms of how you structure the data is in how you will want to use the variable in the heatmap. Maybe thats kind of an arbitrary example trying to make use of a common data set, but I think given that a major contribution of this work is providing a bridge between "tidy" data and heatmaps it would be helpful to have a bit more explanation of how heatmaps and this API relate to the data and its structure. In the paper it states that "Tidy data frames are characterized by having a specific structure where each variable is a column and each observation is a row.". It'd be helpful to expand on how the tidyHeatmap expects "annotation" data to be included in such data and how that fits the 'tidy' paradigm. It might also be useful to compare the tidy representation with representations used in Bioconductor like SummarizedExperiment which in some ways more obviously map to the heatmap visualization.

[stemangiola]

Thanks @AliciaSchep,

Both your issue and feedback have been integrated here https://github.com/stemangiola/tidyHeatmap/pull/44

1) Great suggestion for vdiffr! I wondered how to test images. That has been integrated
2) Good suggestion. I realised that while is genetics the structure element/feature/value/independent variable is almost implicit, this can create confusion generally. I have updated README, vignette and the article, better explaining (also with examples) what the 4+ columns mean.

[stemangiola]

And for commenting on your concern

"Maybe thats kind of an arbitrary example trying to make use of a common data set, but I think given that a major contribution of this work is providing a bridge between "tidy" data and heatmaps it would be helpful to have a bit more explanation of how heatmaps and this API relate to the data and its structure. "

Yes this was totally arbitrary in order to produce an heatmap with annotations. It can be confusing, and I tried to clarify substantially in the README and vignette.

[AliciaSchep]

Incorporating vdiffr addresses my concerns about the tests and the proposed updates to readme look good -- although I would recommend running a spellchecker as I noticed a couple spelling issues. There's an R package that can help with that -- spelling

[stemangiola]

Incorporating vdiffr addresses my concerns about the tests and the proposed updates to readme look good -- although I would recommend running a spellchecker as I noticed a couple spelling issues. There's an R package that can help with that -- spelling

Thanks Alicia, I have updated the pull request with the spell checking framework, and corrected spelling.

[karthik]

Thanks @stemangiola

Let me know once you've merged the spell checks. I found some more (see below) but you may have addressed those in the issues. Be sure to address:

DESCRIPTION does not contain 'Language' field. Defaulting to 'en-US'.

Possible spelling errors in the package

DESCRIPTION does not contain 'Language' field. Defaulting to 'en-US'.
  WORD              FOUND IN
acceots           ifelse_pipe.Rd:5,22
                  ifelse2_pipe.Rd:5,26
annot             annot_to_list.Rd:5,16
centred           scale_robust.Rd:13
circlize          heatmap-methods.Rd:130
colorRamp         heatmap-methods.Rd:130
                  input_heatmap.Rd:32
                  README.md:187
                  README.Rmd:130
colour            description:4
colous            input_heatmap.Rd:32
ComplexHeatmap    add_annotation.Rd:27,30
                  add_bar.Rd:5,17,25
                  add_line.Rd:5,17,25
                  add_point-methods.Rd:7,21,31
                  add_tile-methods.Rd:7,21,31
                  heatmap-methods.Rd:10,152,155,166,171
                  input_heatmap.Rd:36,39,42
                  save_pdf-methods.Rd:31
                  description:2
                  README.md:21
                  README.Rmd:18
                  introduction.Rmd:26
customisation     input_heatmap.Rd:32
df                add_bar-methods.Rd:11
                  add_bar.Rd:10
                  add_line-methods.Rd:11
                  add_line.Rd:10
                  add_point-methods.Rd:14
                  add_tile-methods.Rd:14
                  heatmap-methods.Rd:10,118
                  description:7
dplyr             add_annotation.Rd:30
                  heatmap-methods.Rd:166
                  input_heatmap.Rd:42
                  README.md:157
                  README.Rmd:107
enquo             get_abundance_norm_if_exists.Rd:15
                  get_elements_features_abundance.Rd:27
                  get_elements_features.Rd:19
                  get_elements.Rd:17
                  get_sample_counts.Rd:17
                  get_sample_transcript_counts.Rd:19
                  get_sample_transcript.Rd:17
generalisation    ifelse_pipe.Rd:5,22
                  ifelse2_pipe.Rd:5,26
github            README.md:17
                  README.Rmd:15
ifelse            ifelse_pipe.Rd:5,22
                  ifelse2_pipe.Rd:5,26
InputHeatmap      add_annotation.Rd:16
                  add_bar.Rd:5,17,20
                  add_line.Rd:5,17,20
                  add_point-methods.Rd:7,21,26
                  add_tile-methods.Rd:7,21,26
                  heatmap-methods.Rd:10,155,157,159,161,163,171
io                README.md:17
                  README.Rmd:15
lifecycle         README.md:51
                  README.Rmd:49
                  introduction.Rmd:56
Lifecycle         README.md:6
                  README.Rmd:7
mtcars            README.md:136
                  README.Rmd:89
Pasilla           pasilla.Rd:6,14
pleae             README.md:60
                  README.Rmd:59
quosure           quo_names.Rd:5,16
quosures          quo_names.Rd:10
rlang             prepend.Rd:5,20
sd                scale_robust.Rd:5,16
stemangiola       README.md:17
                  README.Rmd:15
tbl               add_annotation.Rd:30
                  add_bar-methods.Rd:11
                  add_bar.Rd:10
                  add_line-methods.Rd:11
                  add_line.Rd:10
                  add_point-methods.Rd:14
                  add_tile-methods.Rd:14
                  check_if_counts_is_na.Rd:15
                  check_if_duplicated_genes.Rd:19
                  check_if_wrong_input.Rd:17
                  get_x_y_annotation_columns.Rd:10
                  heatmap-methods.Rd:10,118,166,171
                  input_heatmap.Rd:20,42
                  description:7
                  README.md:27
                  README.Rmd:23
                  introduction.Rmd:31
tibble            add_annotation.Rd:22,24,30
                  add_attr.Rd:10,17
                  add_bar-methods.Rd:15
                  add_bar.Rd:14
                  add_class.Rd:10,15
                  add_line-methods.Rd:15
                  add_line.Rd:14
                  add_point-methods.Rd:18
                  add_tile-methods.Rd:18
                  as_matrix.Rd:5,10,20
                  check_if_counts_is_na.Rd:10
                  check_if_duplicated_genes.Rd:10
                  check_if_wrong_input.Rd:10
                  drop_class.Rd:10,15
                  get_abundance_norm_if_exists.Rd:10
                  get_elements_features_abundance.Rd:16
                  get_elements_features.Rd:10
                  get_elements.Rd:10
                  get_sample_counts.Rd:10
                  get_sample_transcript_counts.Rd:10
                  get_sample_transcript.Rd:10
                  heatmap-methods.Rd:166
                  ifelse_pipe.Rd:10,19
                  ifelse2_pipe.Rd:10,23
                  input_heatmap.Rd:42
                  scale_design.Rd:10,15
                  select_closest_pairs.Rd:10,13
                  README.md:9
                  README.Rmd:12
tidybulk          README.md:14
                  README.Rmd:13
tidygate          README.md:12
                  README.Rmd:12
tranform          input_heatmap.Rd:28
transcriptomics   README.md:9
                  README.Rmd:13
viridis           heatmap-methods.Rd:130
Viridis           description:9
                  README.md:30
                  README.Rmd:26
                  introduction.Rmd:33
visualisation     add_bar.Rd:25
                  add_line.Rd:25
                  add_point-methods.Rd:31
                  add_tile-methods.Rd:31

[karthik]

@whedon generate pdf

[whedon]

:point_right: Check article proof :page_facing_up: :point_left:

[karthik]

@whedon check references

[whedon]

Reference check summary:

OK DOIs

- 10.21105/joss.01686 is OK
- 10.1093/bioinformatics/btw313 is OK
- 10.1093/bioinformatics/btu393 is OK

MISSING DOIs

- None

INVALID DOIs

- None

[karthik]

@whedon set version to v1.1.1

[karthik]

@whedon set v1.1.1 as version

[whedon]

OK. v1.1.1 is the version.

[stemangiola]

@whedon generate pdf

[whedon]

:point_right: Check article proof :page_facing_up: :point_left:

[stemangiola]

@whedon generate pdf

[whedon]

:point_right: Check article proof :page_facing_up: :point_left:

[stemangiola]

@karthik Thanks. Done.

[karthik]

@stemangiola Can you please deposit your software in Zenodo and post the DOI here?

[stemangiola]

@whedon generate pdf

[whedon]

:point_right: Check article proof :page_facing_up: :point_left:

[stemangiola]

Hello @karthik

DOI: doi.org/10.5281/zenodo.3966515

[karthik]

@stemangiola Thanks. Can you confirm that the version is now v1.1.2?

[stemangiola]

Yes it is.

[karthik]

@whedon set v1.1.2 as version

[whedon]

OK. v1.1.2 is the version.

[karthik]

@whedon accept

[whedon]

No archive DOI set. Exiting...

[karthik]

@whedon set 10.5281/zenodo.3966515 as archive

[whedon]

OK. 10.5281/zenodo.3966515 is the archive.

[karthik]

@whedon accept

[whedon]

Attempting dry run of processing paper acceptance...

[whedon]

Reference check summary:

OK DOIs

- 10.21105/joss.01686 is OK
- 10.1093/bioinformatics/btw313 is OK
- 10.1093/bioinformatics/btu393 is OK
- 10.1080/10618600.2018.1473780 is OK

MISSING DOIs

- None

INVALID DOIs

- None

[whedon]

:wave: @openjournals/joss-eics, this paper is ready to be accepted and published.

Check final proof :point_right: https://github.com/openjournals/joss-papers/pull/1602

If the paper PDF and Crossref deposit XML look good in https://github.com/openjournals/joss-papers/pull/1602, then you can now move forward with accepting the submission by compiling again with the flag deposit=true e.g.
@whedon accept deposit=true

[stemangiola]

Hello @karthik all good except few packages names seem to have been printed in two lines. magrittr and ComplexHeatmap are splt across lines with no hyphenation.

[karthik]

@whedon generate pdf

[whedon]

:point_right: Check article proof :page_facing_up: :point_left:

[kthyng]

Hi @stemangiola! I will help finishing publishing your paper. I suggest that you change the metadata in your Zenodo archive to exactly match your JOSS paper title and authors.

[stemangiola]

Thanks @kthyng the https://zenodo.org/deposit/3966515 has been updated

[kthyng]

@whedon generate pdf

[whedon]

:point_right: Check article proof :page_facing_up: :point_left:

[kthyng]

@whedon generate pdf

[whedon]

:point_right: Check article proof :page_facing_up: :point_left:

[kthyng]

@whedon accept deposit=true

[whedon]

Doing it live! Attempting automated processing of paper acceptance...

[whedon]

🐦🐦🐦 👉 Tweet for this paper 👈 🐦🐦🐦

[whedon]

🚨🚨🚨 THIS IS NOT A DRILL, YOU HAVE JUST ACCEPTED A PAPER INTO JOSS! 🚨🚨🚨

Here's what you must now do:

1. Check final PDF and Crossref metadata that was deposited :point_right: https://github.com/openjournals/joss-papers/pull/1614
2. Wait a couple of minutes to verify that the paper DOI resolves https://doi.org/10.21105/joss.02472
3. If everything looks good, then close this review issue.

4. Party like you just published a paper! 🎉🌈🦄💃👻🤘

   Any issues? Notify your editorial technical team...

[kthyng]

the doi is still resolving but in the meantime...

Congrats to @stemangiola on your new publication! Thanks to editor @karthik and reviewers @rlbarter and @AliciaSchep — this process wouldn't be possible without your expertise and time.

[whedon]

:tada::tada::tada: Congratulations on your paper acceptance! :tada::tada::tada:

If you would like to include a link to your paper from your README use the following code snippets:

Markdown:
[![DOI](https://joss.theoj.org/papers/10.21105/joss.02472/status.svg)](https://doi.org/10.21105/joss.02472)

HTML:
<a style="border-width:0" href="https://doi.org/10.21105/joss.02472">
  <img src="https://joss.theoj.org/papers/10.21105/joss.02472/status.svg" alt="DOI badge" >
</a>

reStructuredText:
.. image:: https://joss.theoj.org/papers/10.21105/joss.02472/status.svg
   :target: https://doi.org/10.21105/joss.02472

This is how it will look in your documentation:

We need your help!

Journal of Open Source Software is a community-run journal and relies upon volunteer effort. If you'd like to support us please consider doing either one (or both) of the the following:

• Volunteering to review for us sometime in the future. You can add your name to the reviewer list here: https://joss.theoj.org/reviewer-signup.html
• Making a small donation to support our running costs here: https://numfocus.org/donate-to-joss

[kthyng]

Sorry should have left open while waiting for doi

[danielskatz]

@openjournals/dev - for me, the DOI resolves but the PDF does not load in the frame where it should - any thoughts?

[danielskatz]

This is now there!

[whedon]

:tada::tada::tada: Congratulations on your paper acceptance! :tada::tada::tada:

If you would like to include a link to your paper from your README use the following code snippets:

Markdown:
[![DOI](https://joss.theoj.org/papers/10.21105/joss.02472/status.svg)](https://doi.org/10.21105/joss.02472)

HTML:
<a style="border-width:0" href="https://doi.org/10.21105/joss.02472">
  <img src="https://joss.theoj.org/papers/10.21105/joss.02472/status.svg" alt="DOI badge" >
</a>

reStructuredText:
.. image:: https://joss.theoj.org/papers/10.21105/joss.02472/status.svg
   :target: https://doi.org/10.21105/joss.02472

This is how it will look in your documentation:

We need your help!

Journal of Open Source Software is a community-run journal and relies upon volunteer effort. If you'd like to support us please consider doing either one (or both) of the the following:

• Volunteering to review for us sometime in the future. You can add your name to the reviewer list here: https://joss.theoj.org/reviewer-signup.html
• Making a small donation to support our running costs here: https://numfocus.org/donate-to-joss

[stemangiola]

Thanks a lot to everybody for the efforts and time spent here! I will surely add my name to the reviewer list.