Joss-reviews: [REVIEW]: NiBetaSeries: task related correlations in fMRI

[whedon]

Submitting author: @jdkent (James Kent)
Repository: https://github.com/HBClab/NiBetaSeries
Version: v0.3.2
Editor: @arokem
Reviewer: @snastase
Archive: 10.5281/zenodo.3385339

Status

Status badge code:

HTML: <a href="http://joss.theoj.org/papers/a0d2ec4e06309e9b1e21dc302c396290"><img src="http://joss.theoj.org/papers/a0d2ec4e06309e9b1e21dc302c396290/status.svg"></a>
Markdown: [![status](http://joss.theoj.org/papers/a0d2ec4e06309e9b1e21dc302c396290/status.svg)](http://joss.theoj.org/papers/a0d2ec4e06309e9b1e21dc302c396290)

Reviewers and authors:

Please avoid lengthy details of difficulties in the review thread. Instead, please create a new issue in the target repository and link to those issues (especially acceptance-blockers) in the review thread below. (For completists: if the target issue tracker is also on GitHub, linking the review thread in the issue or vice versa will create corresponding breadcrumb trails in the link target.)

Reviewer instructions & questions

@snastase, please carry out your review in this issue by updating the checklist below. If you cannot edit the checklist please:

1. Make sure you're logged in to your GitHub account
2. Be sure to accept the invite at this URL: https://github.com/openjournals/joss-reviews/invitations

The reviewer guidelines are available here: https://joss.theoj.org/about#reviewer_guidelines. Any questions/concerns please let @arokem know.

✨ Please try and complete your review in the next two weeks ✨

Review checklist for @snastase

Conflict of interest

• [x] As the reviewer I confirm that I have read the JOSS conflict of interest policy and that there are no conflicts of interest for me to review this work.

Code of Conduct

• [x] I confirm that I read and will adhere to the JOSS code of conduct.

General checks

• [x] Repository: Is the source code for this software available at the repository url?
• [x] License: Does the repository contain a plain-text LICENSE file with the contents of an OSI approved software license?
• [x] Version: v0.3.2
• [x] Authorship: Has the submitting author (@jdkent) made major contributions to the software? Does the full list of paper authors seem appropriate and complete?

Functionality

• [x] Installation: Does installation proceed as outlined in the documentation?
• [x] Functionality: Have the functional claims of the software been confirmed?
• [x] Performance: If there are any performance claims of the software, have they been confirmed? (If there are no claims, please check off this item.)

Documentation

• [x] A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
• [x] Installation instructions: Is there a clearly-stated list of dependencies? Ideally these should be handled with an automated package management solution.
• [x] Example usage: Do the authors include examples of how to use the software (ideally to solve real-world analysis problems).
• [x] Functionality documentation: Is the core functionality of the software documented to a satisfactory level (e.g., API method documentation)?
• [x] Automated tests: Are there automated tests or manual steps described so that the function of the software can be verified?
• [x] Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support

Software paper

• [x] Authors: Does the paper.md file include a list of authors with their affiliations?
• [x] A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
• [x] References: Do all archival references that should have a DOI list one (e.g., papers, datasets, software)?

[whedon]

Hello human, I'm @whedon, a robot that can help you with some common editorial tasks. @snastase it looks like you're currently assigned as the reviewer for this paper :tada:.

:star: Important :star:

If you haven't already, you should seriously consider unsubscribing from GitHub notifications for this (https://github.com/openjournals/joss-reviews) repository. As a reviewer, you're probably currently watching this repository which means for GitHub's default behaviour you will receive notifications (emails) for all reviews 😿

To fix this do the following two things:

1. Set yourself as 'Not watching' https://github.com/openjournals/joss-reviews:

1. You may also like to change your default settings for this watching repositories in your GitHub profile here: https://github.com/settings/notifications

For a list of things I can do to help you, just type:

@whedon commands

[whedon]

Attempting PDF compilation. Reticulating splines etc...

[whedon]

:point_right: Check article proof :page_facing_up: :point_left:

[arokem]

@snastase : have you had a chance to take a look?

[snastase]

@arokem @jdkent Sorry for losing track of this! Working through it now...

[snastase]

Okay, I think I have a grip on this now—nice contribution and great to see things like this wrapped up into BIDS Apps! To summarize, this project aims to compute beta-series correlations on BIDS-compliant data preprocessed using fMRIPrep. I have some general comments and then a laundry list of smaller-scale suggestions. For the really nit-picky stuff, I can make a PR if you don't mind. Also, bear in mind that I'm more neuroscientist than software developer per se, so apologies if any of these comments are way off the mark!

_General comments:_
First of all, the PDF and introductory documentation (betaseries.html) could be made a little more clear and concise. For example, it wasn't immediately obvious to me what the actual output of the tool is... can I get the actual series of estimated betas? Or only the inter-ROI beta-series correlation matrices? It might be nice to simply get the beta series and forget the correlations (e.g., for a downstream MVPA analysis). Is it absolutely necessary to provide an atlas to define ROIs? If I don't provide an atlas, will it compute beta-series correlations between all voxels (computationally intensive). Basically what I'm trying to say here is that it wasn't obvious what to expect in terms of input–output (namely output), what moving parts are necessary, and how much flexibility there is. I could figure these things out by trial and error, but it seems useful to lay this out a bit more explicitly in the documentation.

I'm a little bit unsure about the Jupyter Notebook-style tutorial walkthrough in the "How to run" documentation. If I was planning to run this, I'd likely be running it from the Linux command line (maybe via a scheduler like Slurm on my server)—not invoking it via Python's subprocess. You jump through a bunch of hoops with Python just to download and modify the example data and only one cell of the tutorial actually runs nibs. I think this material is useful, particularly for users to see how to modify idiosyncratic OpenNeuro datasets, but I'm not sure there's enough focus on the nibs invocation. An alternative approach would be to upload the minimal dataset with all necessary modifications to figshare or something and download that in the tutorial. This would avoid spending so much of the tutorial on data manipulation and spend a few more cells describing the nibs command-line invocation and various options.

This brings up the point that, if the preprocessed BIDS derivatives (e.g., in *_events.tsv) are fairly standard, should we expect nibs be able to handle them internally? For example, you manually rename some columns and reassign "Yes/No" values to 1 and 0 to satisfy assumptions. Another approach to this would be to build in some optional arguments in the nibs CLI that allows the user to specify column names and acceptable value names (and map to numerical values if need be). For example, when I run nibs, I might have a command-line argument specifying that conditions is the column name indicating trial types and that it should have three possible conditions values (neutral, congruent, and incongruent), and another argument that specifying the correct column and mapping {'Yes': 1, 'No': 0}. I'm not necessarily saying this _should_ be the way things are, just offering this an alternative approach. I'm genuinely curious if this would be feasible and if it's better or worse in terms of software design.

_Specific comments and questions:_

• Would it be worth making a Singularity image for this? For example, I'm almost exclusively running fMRIPRep and MRIQC apps on a server via Singularity because I don't have installation privileges. I suppose the alternative is indicating to users to pip install nibetaseries in a conda environment or something along these lines.

• Speaking of installation, if you're degenerate like me and still have a Python 2.7 installation on your machine, pip install nibetaseries will try to install this in 2.7 and wreck. Something like pip3 install nibetaseries or python3 -m pip install nibetaseries works more reliably. I would slightly expand the installation documentation page, specifying Python version, etc.

• It might be worth pointing users to Binder (https://mybinder.org) for running the tutorial Jupyter Notebook interactively.

• I understood that based on the atlas provided, beta series are computed per voxel then averaged across voxels within each ROI? As opposed to averaging time series across voxels then computing the beta series? Is there a reason (or reference) for taking this approach?

• What are the recommendations for high-/low-pass filtering? Is there a precedent in the literature for any recommended values? In fact, the documentation mentions both low- and high-pass filtering, but I only see an option for supplying a low-pass filter in the usage documentation.

• Some of the multiword command-line arguments use "_" and some use "-" ...I would just use underscore in e.g. --atlas-img for consistency with other arguments (e.g., --session_label, --hrf_model)

• Is this backward-compatible with older-style BIDS derivatives from fMRIPrep? E.g., files with and without the "desc-".

• One issue I've encountered with people running apps like fMRIPrep is confusion about the "work" directory; namely, whether it can be safely deleted, whether it should be deleted if re-running from scratch, etc. Would be good to make a note of this in the documentation.

• In the documentation and PDF, I would make it a little more explicit that the "beta" is a colloquial term for the parameter estimates (or regression coefficients) in a GLM.

• It should be made abundantly clear in the documentation that this is running the "LSS" version of the analysis. Are there future plans to allow for optionally running the "LSA" version?

• The “How to run NiBetaSeries” section of the documentation unpacks strangely and doesn’t allow user to scroll down through headings; at first I thought the download links were the only thing there. Clicking on the subheadings in the table of contents, however, brings you to a separate page with the walkthrough. Is there a way to combine these such that the download links simply appear at the top of the same page as the walkthrough?

• Under the "References" heading in the betaseries.html documentation, I would include the full reference text and DOI links.

• Cite the paper for the OpenNeuro dataset you use in the tutorial documentation:
  Verstynen, T. D. (2014). The organization and dynamics of corticostriatal pathways link the medial orbitofrontal cortex to future behavioral responses. _Journal of Neurophysiology, 112_(10), 2457–2469. https://doi.org/10.1152/jn.00221.2014

• I would cite Abdulrahman & Henson (2015) in the PDF.

• I would also cite the BIDS Apps paper in the PDF as this is a BIDS App:
  Gorgolewski, K. J., Alfaro-Almagro, F., Auer, T., Bellec, P., Capotă, M., Chakravarty, M. M., ... & Poldrack, R. A. (2017). BIDS apps: Improving ease of use, accessibility, and reproducibility of neuroimaging data analysis methods. _PLOS Computational Biology, 13_(3), e1005209. https://doi.org/10.1371/journal.pcbi.1005209

• In the PDF, update the fMRIPrep reference by Esteban et al. to the Nature Methods version:
  Esteban, O., Markiewicz, C., Blair, R. W., Moodie, C., Isik, A. I., Erramuzpe, A., Kent, J. D., Goncalves, M., DuPre, E., Snyder, M., Oya, H., Ghosh, S., Wright, J., Durnez, J., Poldrack, R., & Gorgolewski, K. J. (2019). FMRIPrep: a robust preprocessing pipeline for functional MRI. _Nature Methods, 16_, 111–116.

• Code coverage is only 70%... might be worth trying to increase this.

[jdkent]

Thank you so much for the in-depth review @snastase! I will be working on addressing these comments via issues/pull requests this week.

[danielskatz]

👋 @jdkent - What's going on with this submission? Are you working on the comments? Or maybe you have worked on them already, and just need to tell us here?

[jdkent]

Hi @danielskatz, I am working through the comments still. I should have more time to dedicate to the project this week and the next.

[danielskatz]

Thanks for the update.

[labarba]

@jdkent — Can you give us an update? If you're not close to done, please let me know of a time period to set a reminder for you.

[kyleniemeyer]

Hi @jdkent, sorry for the repeated bugging, but any updates?

[jdkent]

Hi Sorry, I was at a conference and I missed responding to the last comment. This is high in my queue, I'm putting in a good faith effort to finish by the end of this week.

[kyleniemeyer]

@whedon remind @jdkent in 1 week

[whedon]

Reminder set for @jdkent in 1 week

[whedon]

:wave: @jdkent, please update us on how things are progressing here.

[jdkent]

Update:
I have at least touched on every issue (except adding a binder link), which I am currently deriving a solution for. Thank you for the comments @snastase, I think the documentation is much improved now. @snastase, could you take another look at the joss issues to see if I adequately covered your comments?

the most up to date documentation should be published on RTD

[snastase]

@jdkent sorry, I'm trying to get to this! Very backed up right now. Hopefully by this weekend.

[arokem]

Hey @snastase : have you had a chance to take another look?

[snastase]

@jdkent sorry the delay—finally got to work back through this. I think this is greatly improved. I went through the documentation and made a PR with minor edits.

I'm satisfied with this but am providing a few additional comments:

1. You might include an equation in the math documentation for LSS to go along with your python pseudocode; e.g.:

Y = X_1\beta_1 + X_n\ne1\beta_n\ne1 + \epsilon,
Y = X_2\beta_2 + X_n\ne2\beta_n\ne2 + \epsilon, ...

1. Important not to conflate "resting-state", which is a task paradigm, with the correlation-based functional connectivity analyses typically run on resting-state data. I tried to make this a bit clearer in the documentation.

2. I confirmed that tutorial runs on Binder. For whatever reason, the link to Binder in on the Tutorials documentation points a kind of off-putting 404 error until the Binder session initializes. (Note also that the References are not rendering properly at the bottom of the Jupyter notebook).

3. Do you really want to introduce Generalized Linear Models in the PDF? Not sure this (and the discussion of e.g. non-normality) is really representative of how GLMs are typically used in fMRI. I would also not say the "peak of the gamma curve is determined by the beta coefficient".

4. I would probably the de-emphasize the importance of the "atlas parcellation" in the PDF. Using an atlas to compute inter-areal correlations is only one particular strategy. For example, you could also compute beta series per voxel and then use ICA to identify "networks" rather than using an a prior parcellation.

5. In general, I would do a final pass through the PDF and make sure it's concise as possible. For example, I would take out unrelated bits like "voxel is shortened form of volumetric pixel" to make the PDF as short and direct as possible.

6. I'm curious why you would compute beta series per voxel then average them within each ROI rather than the much less computationally-intensive approach of averaging the time series in each ROI then computing beta series—but this is not a critical issue.

[arokem]

@jdkent : how are things going? When do you anticipate finishing your current revisions?

[jdkent]

I hope to have the current revisions done by Friday this week.

[jdkent]

@arokem, I believe I have responded to all of @snastase's comments, and have either done revisions or provided reasons why the change is not happening yet, while leaving the issue open.

@whedon generate pdf

[arokem]

Great! @snastase : could you please confirm that everything has been addressed? I see that all the check-boxes are checked, which means that this is probably ready to go.

[arokem]

@whedon generate pdf

[whedon]

Attempting PDF compilation. Reticulating splines etc...

[whedon]

:point_right: Check article proof :page_facing_up: :point_left:

[arokem]

@whedon check references

[whedon]

Attempting to check references...

[whedon]

```Reference check summary:

OK DOIs

• 10.3389/fnsys.2015.00126 is OK
• 10.1038/s41592-018-0235-4 is OK
• 10.1016/j.neuroimage.2004.06.035 is OK
• 10.1016/j.neuroimage.2011.08.076 is OK
• 10.1016/j.neuroimage.2015.11.009 is OK
• 10.1016/j.neuroimage.2012.05.057 is OK
• 10.1371/journal.pcbi.1005209 is OK
• 10.3389/fnsys.2010.00008 is OK
• 10.1371/journal.pone.0013701 is OK

MISSING DOIs

• None

INVALID DOIs

• None
  ```

[snastase]

Great! @snastase : could you please confirm that everything has been addressed? I see that all the check-boxes are checked, which means that this is probably ready to go.

@arokem Good to go on my end! @jdkent has addressed my comments.

[arokem]

Great! Thanks for all your work on this review, @snastase. I concur on my end.

At this point, @jdkent : could you please create an archive of your repository (e.g., using Zenodo) and then post the DOI here. When creating the archive, please make sure that:

• The title of the archive is the same as the JOSS paper title
• The authors of the archive are the same as the JOSS paper authors

[jdkent]

@whedon generate pdf

[whedon]

Attempting PDF compilation. Reticulating splines etc...

[whedon]

:point_right: Check article proof :page_facing_up: :point_left:

[jdkent]

@whedon check references

[whedon]

Attempting to check references...

[whedon]

```Reference check summary:

OK DOIs

• 10.3389/fnsys.2015.00126 is OK
• 10.1038/s41592-018-0235-4 is OK
• 10.1016/j.neuroimage.2004.06.035 is OK
• 10.1016/j.neuroimage.2011.08.076 is OK
• 10.1016/j.neuroimage.2015.11.009 is OK
• 10.1016/j.neuroimage.2012.05.057 is OK
• 10.1371/journal.pcbi.1005209 is OK
• 10.3389/fnsys.2010.00008 is OK
• 10.1371/journal.pone.0013701 is OK

MISSING DOIs

• None

INVALID DOIs

• None
  ```

[jdkent]

@arokem here is the doi: 10.5281/zenodo.3382330

I had some trouble with zenodo, but I think I have it all worked out, the page is here

I second that big thanks to @snastase! and thank you @arokem for keeping us moving along!

[arokem]

Hi @jdkent : I see that the Zenodo submission has two authors (you and Peer), but you are the sole author of the paper. That needs to be consistent. I believe that you should be able to edit the Zenodo submission metadata.

[jdkent]

I remade the paper with Peer as a co-author, or were you referencing another document?

[arokem]

OK. I see that he approved that in https://github.com/HBClab/NiBetaSeries/pull/181.

[arokem]

@whedon set 10.5281/zenodo.3382330 as archive

[whedon]

OK. 10.5281/zenodo.3382330 is the archive.

[arokem]

@openjournals/joss-eics : I believe that this is now ready for your review

[labarba]

@whedon accept

[whedon]

Attempting dry run of processing paper acceptance...

[whedon]

Check final proof :point_right: https://github.com/openjournals/joss-papers/pull/939

If the paper PDF and Crossref deposit XML look good in https://github.com/openjournals/joss-papers/pull/939, then you can now move forward with accepting the submission by compiling again with the flag deposit=true e.g.
@whedon accept deposit=true

[labarba]

I see that the latest version tagged on the GitHub repo is v0.3.0, while here we have v0.2.3. Please confirm that we should change the version here to match the tag in the repo. Also, it is marked as a "Pre-release." Are you sure you want that? It generally means the release is unstable—if that were the case, you would not be publishing a paper about the software at this time.

Since I'm sending this back to you with release questions, might as well add minor edits: please capitalize Python and Matlab, use commas after "e.g." and "i.e." and make one final pass at conciseness. For example, you can delete "With the concept of beta under our belt" (excess clutter). The phrase "Neurohackademy is to thank" is grammatically awkward. I'm also a bit unsatisfied with your headings: Introduction, Background, Overview. It seems all headings in the paper could be shuffled around without harm to the narrative, which makes me think they are superfluous. Perhaps just one Background heading and one Software Overview? (We sometimes also see Statement of Need, given that this is one of the requested topics in our docs. Give it a thought.

[jdkent]

Thanks for noting the pre-release check, that was a mistake.

Thank you for the feedback @labarba, I believe I've addressed your comments in the referenced pull request (HBClab/NiBetaSeries#185)

Note, I've merged in one pull request since the making the zenodo doi, with another on right on it's heels, so I could just do one more minor release and say v0.3.1 will be the JOSS release when merging HBClab/NiBetaSeries#185.

The alternative as I see it is to modify the v0.3.0 release, either way, I would like to change the version from v0.2.3.

Let me know if you have any preferences for modifying v0.3.0 or cutting a new patch release (v0.3.1) for the JOSS submission.

Thanks!

[labarba]

Feel free to increase the patch release, and report the final version number here, so we can add it.

[jdkent]

@whedon generate pdf

[whedon]

Attempting PDF compilation. Reticulating splines etc...

[whedon]

:point_right: Check article proof :page_facing_up: :point_left:

[jdkent]

@whedon generate pdf

[whedon]

Attempting PDF compilation. Reticulating splines etc...

[whedon]

:point_right: Check article proof :page_facing_up: :point_left:

[jdkent]

• The final version number is v0.3.2
• here is the doi for the release: 10.5281/zenodo.3385339.
• Only Peer and I are authors, but I did include everyone as "contributors". Is that okay?

[jdkent]

Hi @labarba, I believe the repository is ready pending the points above.

[arokem]

@openjournals/joss-eics : is everything in order here?

[labarba]

I'm not on rotation, but could you now check that the issues I flagged were resolved (version, etc.), and update as needed with whedon?
☝️ @arokem

[arokem]

@whedon set version as v0.3.2

[arokem]

@whedon set archive as 10.5281/zenodo.3385339

[arokem]

Either I'm having some syntax errors, or @whedon is taking a nap.

[arfon]

@whedon set 10.5281/zenodo.3385339 as archive

[whedon]

OK. 10.5281/zenodo.3385339 is the archive.

[arfon]

@whedon set v0.3.2 as version

[whedon]

OK. v0.3.2 is the version.

[arfon]

@arokem ☝️

[arokem]

Thanks! 🤦‍♂

[Kevin-Mattheus-Moerman]

@whedon generate pdf

[whedon]

Attempting PDF compilation. Reticulating splines etc...

[whedon]

:point_right: Check article proof :page_facing_up: :point_left:

[Kevin-Mattheus-Moerman]

@arfon the link to the archive in the PDF does not seem to work. Is this normal at this point?

[Kevin-Mattheus-Moerman]

@jdkent can you work on/check the following for your paper:

• [x] We are about to process your paper for acceptance. Can you please double check the paper thoroughly and make any final changes (refer to them briefly here if you make any changes). Also ensure the author names and the acknowledgements (if any), and links to funding sources etc, are accurate.
• [x] Can you check your affiliation. Is "Neuroscience Program" correct? Usually people list a department rather than a program (also it should perhaps be: "Interdisciplinary Graduate Program in Neuroscience"?), should it perhaps be "Psychological and Brain Sciences" instead, like in your profile page?
• [x] Can you make the affiliations more complete by adding the city and country as well please?
• [x] Consider rephrasing the following (which sounds awkward) "Three dimensions are brain voxels,..." to something like "The first three dimensions represent the voxel dimensions,... "
• [x] Insert missing "be" in this sentence: "For many intents and purposes, the 4D beta series can [be] analyzed...."
• [x] When finished making the above changes please run @whedon generate pdf to update your proof

[Kevin-Mattheus-Moerman]

@arokem could you please verify the above once @jdkent has worked on them, and ping me when we can proceed to the next step? Thanks

[arfon]

@arfon the link to the archive in the PDF does not seem to work. Is this normal at this point?

The link to the archive works in the final proofs (when you do @whedon accept)

[jdkent]

@Kevin-Mattheus-Moerman, thank you for your suggestions! For making the proof, is it alright if the paper on zenodo/GitHub tag is slightly different than the proof that is used in the journal?

I can email zenodo and ask them to update the repository and I can modify which commit v0.3.2 points to (but I cannot change the pypi release).

However, it would be easier if I could edit the JOSS paper on master and have the git tag and zenodo archive contain a slightly different JOSS paper.

[Kevin-Mattheus-Moerman]

@jdkent It is okay that the Zenodo archived paper is different from this one. The purpose of the archive is to capture the software. Ping me here when you've implemented the changes.

[jdkent]

@arokem could you check this pull request to make sure I covered the above points? Thanks!

(also it should perhaps be: "Interdisciplinary Graduate Program in Neuroscience"?), should it perhaps be "Psychological and Brain Sciences" instead, like in your profile page?

I'm keeping the Interdisciplinary Graduate Program in Neuroscience as my affiliation since that is what my PhD will say when I graduate.

[arokem]

@whedon render pdf

[arokem]

@whedon generate pdf

[whedon]

Attempting PDF compilation. Reticulating splines etc...

[whedon]

:point_right: Check article proof :page_facing_up: :point_left:

[arokem]

Oh - of course I don't see these changes in the PDF. They're on a separate branch.

[arokem]

Looks alright to me.

[jdkent]

@whedon generate pdf

[whedon]

Attempting PDF compilation. Reticulating splines etc...

[whedon]

:point_right: Check article proof :page_facing_up: :point_left:

[jdkent]

@Kevin-Mattheus-Moerman

From this commit:

• I changed a couple of the sentences to avoid awkward phrasing and the sentence missing "be".
• I added city/country to affiliations and I made my affiliation more explicit
• I added and verified links to funding sources.

Let me know if I've covered all your suggestions adequately.

[Kevin-Mattheus-Moerman]

@whedon check references

[whedon]

Attempting to check references...

[whedon]

```Reference check summary:

OK DOIs

• 10.3389/fnsys.2015.00126 is OK
• 10.1038/s41592-018-0235-4 is OK
• 10.1016/j.neuroimage.2004.06.035 is OK
• 10.1016/j.neuroimage.2011.08.076 is OK
• 10.1016/j.neuroimage.2015.11.009 is OK
• 10.1016/j.neuroimage.2012.05.057 is OK
• 10.1371/journal.pcbi.1005209 is OK
• 10.3389/fnsys.2010.00008 is OK
• 10.1371/journal.pone.0013701 is OK

MISSING DOIs

• None

INVALID DOIs

• None
  ```

[Kevin-Mattheus-Moerman]

@jdkent thanks, this looks good now.

[Kevin-Mattheus-Moerman]

@whedon accept

[whedon]

Attempting dry run of processing paper acceptance...

[whedon]

Check final proof :point_right: https://github.com/openjournals/joss-papers/pull/966

If the paper PDF and Crossref deposit XML look good in https://github.com/openjournals/joss-papers/pull/966, then you can now move forward with accepting the submission by compiling again with the flag deposit=true e.g.
@whedon accept deposit=true

[Kevin-Mattheus-Moerman]

@whedon accept deposit=true

[whedon]

Doing it live! Attempting automated processing of paper acceptance...

[whedon]

🐦🐦🐦 👉 Tweet for this paper 👈 🐦🐦🐦

[whedon]

🚨🚨🚨 THIS IS NOT A DRILL, YOU HAVE JUST ACCEPTED A PAPER INTO JOSS! 🚨🚨🚨

Here's what you must now do:

1. Check final PDF and Crossref metadata that was deposited :point_right: https://github.com/openjournals/joss-papers/pull/967
2. Wait a couple of minutes to verify that the paper DOI resolves https://doi.org/10.21105/joss.01295
3. If everything looks good, then close this review issue.

4. Party like you just published a paper! 🎉🌈🦄💃👻🤘

   Any issues? notify your editorial technical team...

[Kevin-Mattheus-Moerman]

@jdkent congratulations on your publication in JOSS. Thank you @arokem for editing this submission and thank you @snastase for reviewing this work. :tada:

[whedon]

:tada::tada::tada: Congratulations on your paper acceptance! :tada::tada::tada:

If you would like to include a link to your paper from your README use the following code snippets:

Markdown:
[![DOI](https://joss.theoj.org/papers/10.21105/joss.01295/status.svg)](https://doi.org/10.21105/joss.01295)

HTML:
<a style="border-width:0" href="https://doi.org/10.21105/joss.01295">
  <img src="https://joss.theoj.org/papers/10.21105/joss.01295/status.svg" alt="DOI badge" >
</a>

reStructuredText:
.. image:: https://joss.theoj.org/papers/10.21105/joss.01295/status.svg
   :target: https://doi.org/10.21105/joss.01295

This is how it will look in your documentation:

We need your help!

Journal of Open Source Software is a community-run journal and relies upon volunteer effort. If you'd like to support us please consider doing either one (or both) of the the following:

• Volunteering to review for us sometime in the future. You can add your name to the reviewer list here: https://joss.theoj.org/reviewer-signup.html
• Making a small donation to support our running costs here: https://numfocus.salsalabs.org/donate-to-joss

[jdkent]

@snastase - thanks again for your review!

@labarba & @Kevin-Mattheus-Moerman - thank you for helping making the text clearer and answering my questions!

@arokem - thank you for your guidance during this process!

This was a great experience!