Submitting author: @ManuelaS ()
Repository: https://bitbucket.org/manuela_s/hcp/
Version: v1.0
Editor: @Kevin-Mattheus-Moerman
Reviewers: @piermorel
Archive: 10.5281/zenodo.3242593
Status badge code:
HTML: <a href="http://joss.theoj.org/papers/6a080ea92e93543e9f8a787ea99d228a"><img src="http://joss.theoj.org/papers/6a080ea92e93543e9f8a787ea99d228a/status.svg"></a>
Markdown: [](http://joss.theoj.org/papers/6a080ea92e93543e9f8a787ea99d228a)
Reviewers and authors:
Please avoid lengthy details of difficulties in the review thread. Instead, please create a new issue in the target repository and link to those issues (especially acceptance-blockers) in the review thread below. (For completists: if the target issue tracker is also on GitHub, linking the review thread in the issue or vice versa will create corresponding breadcrumb trails in the link target.)
@trallard & @piermorel, please carry out your review in this issue by updating the checklist below. If you cannot edit the checklist please:
The reviewer guidelines are available here: https://joss.theoj.org/about#reviewer_guidelines. Any questions/concerns please let @Kevin-Mattheus-Moerman know.
✨ Please try and complete your review in the next two weeks ✨
paper.md
file include a list of authors with their affiliations?paper.md
file include a list of authors with their affiliations?Hello human, I'm @whedon, a robot that can help you with some common editorial tasks. @trallard, it looks like you're currently assigned as the reviewer for this paper :tada:.
:star: Important :star:
If you haven't already, you should seriously consider unsubscribing from GitHub notifications for this (https://github.com/openjournals/joss-reviews) repository. As a reviewer, you're probably currently watching this repository which means for GitHub's default behaviour you will receive notifications (emails) for all reviews 😿
To fix this do the following two things:
For a list of things I can do to help you, just type:
@whedon commands
Attempting PDF compilation. Reticulating splines etc...
@ManuelaS, @trallard, @piermorel we've started the review process here. Let me know if you have any questions.
@trallard, @piermorel can you give an indication as to when you are able to work on this review? Thanks again for your help! Let me know if you have any questions.
Hey @Kevin-Mattheus-Moerman I am OOO until the beginning of April so will move this then
Hi, thanks for the reminder. I'll get on it by the end of the week!
Hi @ManuelaS I would like to report a few issues on your repository for some problems with the code but I can't find a way to create issues on your bitbucket repository. I will make them here for now :
Error using HCP.Heatmap (line 177)
'has_feature_dendrogram' is not a recognized parameter.
It works OK without that argruntests('HCP.tests.HeatmapCovariatePlotTest')
fails on all tests with the following message Undefined function 'clustergram' for input arguments of type 'cell'.
. I'm on R2017a on a Mac.Hi @piermorel,
Thanks for reviewing the HCP package!
I would like to report a few issues on your repository for some problems with the code but I can't find a way to create issues on your bitbucket repository.
I changed the permissions in the settings for the issue tracker to allow for public and anonymous issues to be created, let me know if you run into problems.
- The example code on repo webpage is not fully functional :
Error using HCP.Heatmap (line 177) 'has_feature_dendrogram' is not a recognized parameter.
It works OK without that arg
This was a typo in the parameter name in the ReadMe. It should have read 'show_feature_dendrogram' instead. I have checked in a fix for the ReadMe in commit 46da0cc.
- Is the window resizing supposed to behave this way ? (I'm just getting widening of the plotted elements but it stays always the same height).
Yes, this is the intended behavior. The height of the figure is controlled with the 'covariate_row_height' and 'covariate_row_margin' parameters of the 'HeatmapCovariatePlot' constructor and 'height_per_feature' parameter of the 'Heatmap' constructor for setting properties for the metadata annotations and heatmap features, respectively.
runtests('HCP.tests.HeatmapCovariatePlotTest')
fails on all tests with the following messageUndefined function 'clustergram' for input arguments of type 'cell'.
. I'm on R2017a on a Mac.
A subset of the unit tests check the behavior of HCP against Matlab's built-in clustegram which requires the Bioinformatics toolbox. I do not understand why all the tests are failing, rather than just the 5 tests that use clustergram. Do you mind posting the full output from the tests?
Regardless, I moved the tests that depend on clustergram to a separate class and documented the dependency in the ReadMe (commit 6be49b6).
@trallard, @piermorel can you update me on progress or resume review at this point? Thanks!
Hi @Kevin-Mattheus-Moerman I was ooo until yesterday so I will do my best to catch-up with this in the next couple of days
Hi @trallard , @piermorel & @Kevin-Mattheus-Moerman, is there anything I can do to drive this forward? Thanks a million!
@whedon check references
Attempting to check references...
@whedon generate pdf
Attempting PDF compilation. Reticulating splines etc...
@ManuelaS can this package be installed / used through octave? My guess is it does not as it needs the Statistics toolbox but thought I would check
Created the following issue regarding the statement of need point https://bitbucket.org/manuela_s/hcp/issues/1/joss-review-statement-of-need
@whedon generate pdf
Attempting PDF compilation. Reticulating splines etc...
@ManuelaS can this package be installed / used through octave? My guess is it does not as it needs the Statistics toolbox but thought I would check
@trallard , no the package can't be used with Octave. It makes extensive use of MATLAB tables and the Statistics toolbox neither of which would work.
Created the following issue regarding the statement of need point https://bitbucket.org/manuela_s/hcp/issues/1/joss-review-statement-of-need
@trallard, I added clearer statement of need for HCP and listed alternative (or lack of in MATLAB) and R. This information was included in the paper, however I agree it makes it easier for potential users if it is also included in the ReadMe. Please, re-open the issue if you don't think this fully addresses your comment.
Note that whedon is failing to generate the pdf. @arfon explained why and posted the pdf in the pre-review thread https://github.com/openjournals/joss-reviews/issues/1206#issuecomment-457930985
@trallard, @piermorel, @ManuelaS can you provide an update on progress for this review?
Hi @Kevin-Mattheus-Moerman with the chance in institutions / jobs recently I only discovered that now my MATLAB license is no longer valid. So I am afraid I cannot longer complete the rest of my review unfortunately, perhaps @piermorel can proceed?
Thanks for following up @Kevin-Mattheus-Moerman.
I have worked through all the issues raised so far by @trallard and @piermorel and addressed them either here in the review thread or directly in repo issues.
Let me know if I can help with anything to move forward with this review and thanks @trallard and @piermorel for the all the suggestions to improve HCP so far!
Hi @Kevin-Mattheus-Moerman with the chance in institutions / jobs recently I only discovered that now my MATLAB license is no longer valid. So I am afraid I cannot longer complete the rest of my review unfortunately, perhaps @piermorel can proceed?
@trallard would you be able to to use a MATLAB trial license (valid for 30 days, can be extended) to complete the review? I think if you have it already installed you'd only need to update your license number with a trial license number so this could be fast. Please let me know what you decide, your help is greatly appreciated.
Hi @Kevin-Mattheus-Moerman I will try and get the trial license working, if anything fails during the process I'll let you know so that I do not stall the review any further
@trallard thank you so much for the extra effort involved!
Hi everyone, I'm sorry for the delay in getting back on the review. I'm still here !
```Reference check summary:
OK DOIs
MISSING DOIs
INVALID DOIs
Thanks for your patience. I could install without any problems, run the tests and example code. I checked all the points in the review checklist but two because I guess it depends on JOSS' guidelines and way of doing things :
One last comment/suggestion, which is not review-breaking but could be good for convenience : due to the class and subclass hierarchy of HCP, it is quite hard to find the doc of the useful methods/parameters just by following we get by doing doc HCP
as suggested in readme (one has to "know" the meaty part is in HCP.BlockContainer for example). The usage section and examples are clear enough and one can do doc methodname
... Maybe this could be made clear in the reference documentation section: there are actually few methods that the user should use if I'm not mistaken so they could be spelled out. Once the correct page has been reached the documentation is good.
@whedon generate pdf
Attempting PDF compilation. Reticulating splines etc...
@whedon generate pdf
Attempting PDF compilation. Reticulating splines etc...
Hi folks 👋🏽
Unfortunately I could not get a fresh MATLAB install working on. So I'll have to pull out of this review as I would not like to hold this any longer
@whedon remove @trallard as reviewer
OK, @trallard is no longer a reviewer
Okay thanks for your help so far @trallard
@arfon it seems whedon does not process the PDF or is stuck. Do you know what's going on?
@Kevin-Mattheus-Moerman - the issue is with the bibtex entry names. Pandoc can't handle entries like @U.S.DepartmentofCommerce1977
, @U.S.DepartmentofCommerce1977a
, @CynthiaA.Brewer
. (doesn't like the periods). Please rename these bibtex entries.
Also, while I was looking at the paper.md
, I noticed that there are links like this in the paper:
Source data for the figure are further detailed in [occupation_by_us_state.md](../+examples/data/occupation_by_us_state.md)
While these work fine on GitHub, they won't work in the final PDF as they're relative links. @ManuelaS - can you update these to be full URLs?
- For the references, one of the dataset has multiple similar references that make the list very long, and look weird in the compiled PDF. Not sure if this is a problem on your side or on the compilation side.
Thanks @piermorel for a thorough reviewing theHCP
package!
We combined all the KFF references into a single reference and kept the list of the individual data sources we used in occupation_by_us_state.md as we would like to make sure all the data sources are credited fully. @Kevin-Mattheus-Moerman, is this an acceptable approach?
- For the version, comments that were made by @trallard and myself resulted in code/readme changes and the original release v0.1 is not up-to-date anymore. Should a new version be created?
Our understanding is that we should create a new release (and a zenodo archive) once all the feedback from the reviewers and editor has been addressed
One last comment/suggestion, which is not review-breaking but could be good for convenience : due to the class and subclass hierarchy of HCP, it is quite hard to find the doc of the useful methods/parameters just by following we get by doing
doc HCP
as suggested in readme (one has to "know" the meaty part is in HCP.BlockContainer for example). The usage section and examples are clear enough and one can dodoc methodname
... Maybe this could be made clear in the reference documentation section: there are actually few methods that the user should use if I'm not mistaken so they could be spelled out. Once the correct page has been reached the documentation is good.
Thanks for this suggestion. We have updated the package documentation to include a workflow section and additional "See also" links directly to the most relevant methods. We have also extended the HCP.HeatmapCovariatePlot
documentation with "See also" links to HCP.BlockContainer
, as it was not obvious how to make use of the blocks property.
Thanks @arfon for looking into this!
@Kevin-Mattheus-Moerman - the issue is with the bibtex entry names. Pandoc can't handle entries like
@U.S.DepartmentofCommerce1977
,@U.S.DepartmentofCommerce1977a
,@CynthiaA.Brewer
. (doesn't like the periods). Please rename these bibtex entries.
Done.
Also, while I was looking at the
paper.md
, I noticed that there are links like this in the paper:Source data for the figure are further detailed in [occupation_by_us_state.md](../+examples/data/occupation_by_us_state.md)
While these work fine on GitHub, they won't work in the final PDF as they're relative links. @ManuelaS - can you update these to be full URLs?
Done.
@whedon generate pdf
Attempting PDF compilation. Reticulating splines etc...
Seems like whedon still has issues generating the pdf (maybe still because of the big images).
I am attaching the pdf as generated by running pandoc locally:
HCP_paper.pdf
The pdf may look slightly different from the official one that whedon will generate.
It could be that, plus the large author lists sometimes makes Whedon run out of memory on Heroku. I've compiled a version here: 10.21105.joss.01291.pdf
Hi @ManuelaS, the doc changes are nice ! If the references issues are fine with editors that's all good for me.
@ManuelaS in your paper you state
HCP plotting functionality has been applied in a scholarly manuscript currently undergo-
ing revisions...
Has any citable object been created for this work? Would you consider uploading a pre-print and citing the pre-print in this paper? (e.g. with https://engrxiv.org/)
@arfon the Cobeldick
reference contains a URL which is too long. Is there a way to prevent it from spilling out of the margin?
Also the reference to The Kaiser Family Foundation. (2019)
seems to create quite a large section in the paper reference list, perhaps it lists the entire describtion or "abstract" for this website. Do you think this can just be fixed by deleting this from the .bib file?
@Kevin-Mattheus-Moerman - yes, I'd recommend removing the abstract
field from that bibtex entry.
Is there a way to prevent it from spilling out of the margin?
Our LaTeX template is a little fussy and inconsistent with this and I don't have a robust fix sorry. I can try fixing this locally once the paper is ready to accept.
@ManuelaS in your paper you state
HCP plotting functionality has been applied in a scholarly manuscript currently undergo-
ing revisions...Has any citable object been created for this work? Would you consider uploading a pre-print and citing the pre-print in this paper? (e.g. with https://engrxiv.org/)
@Kevin-Mattheus-Moerman , the package is being used in research work, but the work is not yet citable and not available as pre-print. Should I amend "HCP plotting functionality has been applied in a scholarly manuscript currently undergoing revisions and in exploratory analyses in several other ongoing research projects." to read "HCP plotting functionality has been applied in exploratory analyses in several ongoing research projects." in paper.md?"
@ManuelaS No need to amend the text. I do recommend that you upload your manuscript as a pre-print (e.g. your submitted version prior to review), as most journals allow this (search for your journal's policies here: http://sherpa.ac.uk/romeo/index.php), and to cite the pre-print DOI in the paper. If you are not able to for some reason we'll leave it as is.
Also the reference to
The Kaiser Family Foundation. (2019)
seems to create quite a large section in the paper reference list, perhaps it lists the entire describtion or "abstract" for this website. Do you think this can just be fixed by deleting this from the .bib file?
The reference to The Kaiser Family Foundation. (2019) is very long because it covers several individual datasets downloaded from the KFF website that we used in the second case study.
If I interpret correctly the KFF citation policy for "State Health Facts", the citation should include both the KFF website and the original data source, which differs for each dataset. In the original paper.md, we used
to include a separate citation for each source data which made the caption for Figure 2 very long as pointed out by @piermorel. We combined the citation for the KFF website with the citations for each data source
in a single citation (commit 75b7835) to make the manuscript more readable and kept all single and detailed citations spelled out in occupation_by_us_state.md
If you prefer a different way of handing this citation, I would be happy to change it.
@openjournals/jose-editors the above sounds reasonable. Any thoughts on this very long reference listing? See paper here: https://github.com/openjournals/joss-reviews/files/3248049/10.21105.joss.01291.pdf
@ManuelaS No need to amend the text. I do recommend that you upload your manuscript as a pre-print (e.g. your submitted version prior to review), as most journals allow this (search for your journal's policies here: http://sherpa.ac.uk/romeo/index.php), and to cite the pre-print DOI in the paper. If you are not able to for some reason we'll leave it as is.
Thanks @Kevin-Mattheus-Moerman for the suggestion, but we are not able to submit it to pre-print at this time.
No suggestions on the long reference, except to use caps protection {}
, for example, around U.S
.
You also need to do something to break the long URL in Cobeldick, S. (2019). ColorBrewer (or create a short URL).
@whedon generate pdf
Attempting PDF compilation. Reticulating splines etc...
@arfon it looks like we might need help creating the paper again.
@Kevin-Mattheus-Moerman, I have added caps protection for the references and replaced the long Cobeldick reference link with a shortened URL. I am attaching an updated pdf of the paper generated locally since whedon still fails to generate it.
Here's a compiled version from my local Whedon: 10.21105.joss.01291.pdf
@whedon check references
Attempting to check references...
```Reference check summary:
OK DOIs
MISSING DOIs
INVALID DOIs
@ManuelaS can you check the above references for DOI's? Thanks.
@ManuelaS can you check the above references for DOI's? Thanks.
@Kevin-Mattheus-Moerman, it looks like the 2 references that whedon identified as missing the DOI do not have a "valid" DOI:
@ManuelaS okay thanks for checking
@ManuelaS at this point can you please do the following:
Hi @Kevin-Mattheus-Moerman,
- [x] Can you archive a copy of the reviewed software in ZENODO? Can you list the DOI of the archived version here when you have it? Make sure the ZENODO archive title matches your paper title and that the author list matches with the paper as well.
Done. The archive is 10.5281/zenodo.3242593
- [x] Can you provide the version tag for the reviewed/archived package (it may have changed during review)?
Done. Version tag v1.0
@whedon set 10.5281/zenodo.3242593 as archive
OK. 10.5281/zenodo.3242593 is the archive.
@whedon set v1.0 as version
OK. v1.0 is the version.
@openjournals/joss-eics this submission is ready to be accepted
@arfon I think we resolved the long hyperlink issue but pinging you here just in case you need to check the PDF for this one again.
@piermorel @trallard - many thanks for your reviews here and to @Kevin-Mattheus-Moerman for editing this submission ✨
@ManuelaS - your paper is now accepted into JOSS :zap::rocket::boom:
:tada::tada::tada: Congratulations on your paper acceptance! :tada::tada::tada:
If you would like to include a link to your paper from your README use the following code snippets:
Markdown:
[](https://doi.org/10.21105/joss.01291)
HTML:
<a style="border-width:0" href="https://doi.org/10.21105/joss.01291">
<img src="http://joss.theoj.org/papers/10.21105/joss.01291/status.svg" alt="DOI badge" >
</a>
reStructuredText:
.. image:: http://joss.theoj.org/papers/10.21105/joss.01291/status.svg
:target: https://doi.org/10.21105/joss.01291
This is how it will look in your documentation:
We need your help!
Journal of Open Source Software is a community-run journal and relies upon volunteer effort. If you'd like to support us please consider doing either one (or both) of the the following:
Thanks to @arfon, @Kevin-Mattheus-Moerman , @piermorel and @trallard for making HCP better - sincerely appreciate the experience!
Most helpful comment
Hi everyone, I'm sorry for the delay in getting back on the review. I'm still here !