Joss-reviews: [REVIEW]: CITapp - a free reaction time-based Concealed Information Test application

[whedon]

Submitting author: @gasparl (Gáspár Lukács)
Repository: https://github.com/gasparl/citapp_pc
Version: 1.0.3
Editor: @yochannah
Reviewer: @vsoch, @andytwoods
Archive: 10.5281/zenodo.2561206

Status

Status badge code:

HTML: <a href="http://joss.theoj.org/papers/b2d9ee93da00cb5b3fdc962d50df5063"><img src="http://joss.theoj.org/papers/b2d9ee93da00cb5b3fdc962d50df5063/status.svg"></a>
Markdown: [![status](http://joss.theoj.org/papers/b2d9ee93da00cb5b3fdc962d50df5063/status.svg)](http://joss.theoj.org/papers/b2d9ee93da00cb5b3fdc962d50df5063)

Reviewers and authors:

Please avoid lengthy details of difficulties in the review thread. Instead, please create a new issue in the target repository and link to those issues (especially acceptance-blockers) in the review thread below. (For completists: if the target issue tracker is also on GitHub, linking the review thread in the issue or vice versa will create corresponding breadcrumb trails in the link target.)

Reviewer instructions & questions

@vsoch & @andytwoods, please carry out your review in this issue by updating the checklist below. If you cannot edit the checklist please:

1. Make sure you're logged in to your GitHub account
2. Be sure to accept the invite at this URL: https://github.com/openjournals/joss-reviews/invitations

The reviewer guidelines are available here: https://joss.theoj.org/about#reviewer_guidelines. Any questions/concerns please let @yochannah know.

✨ Please try and complete your review in the next two weeks ✨

Review checklist for @vsoch

Conflict of interest

• [x] As the reviewer I confirm that I have read the JOSS conflict of interest policy and that there are no conflicts of interest for me to review this work.

Code of Conduct

• [x] I confirm that I read and will adhere to the JOSS code of conduct.

General checks

• [x] Repository: Is the source code for this software available at the repository url?
• [x] License: Does the repository contain a plain-text LICENSE file with the contents of an OSI approved software license?
• [x] Version: 1.0.3
• [x] Authorship: Has the submitting author (@gasparl) made major contributions to the software? Does the full list of paper authors seem appropriate and complete?

Functionality

• [x] Installation: Does installation proceed as outlined in the documentation?
• [x] Functionality: Have the functional claims of the software been confirmed?
• [x] Performance: If there are any performance claims of the software, have they been confirmed? (If there are no claims, please check off this item.)

Documentation

• [x] A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
• [x] Installation instructions: Is there a clearly-stated list of dependencies? Ideally these should be handled with an automated package management solution.
• [x] Example usage: Do the authors include examples of how to use the software (ideally to solve real-world analysis problems).
• [x] Functionality documentation: Is the core functionality of the software documented to a satisfactory level (e.g., API method documentation)?
• [x] Automated tests: Are there automated tests or manual steps described so that the function of the software can be verified?
• [x] Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support

Software paper

• [x] Authors: Does the paper.md file include a list of authors with their affiliations?
• [x] A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
• [x] References: Do all archival references that should have a DOI list one (e.g., papers, datasets, software)?

Review checklist for @andytwoods

Conflict of interest

• [x] As the reviewer I confirm that I have read the JOSS conflict of interest policy and that there are no conflicts of interest for me to review this work.

Code of Conduct

• [x] I confirm that I read and will adhere to the JOSS code of conduct.

General checks

• [x] Repository: Is the source code for this software available at the repository url?
• [x] License: Does the repository contain a plain-text LICENSE file with the contents of an OSI approved software license?
• [x] Version: 1.0.3
• [x] Authorship: Has the submitting author (@gasparl) made major contributions to the software? Does the full list of paper authors seem appropriate and complete?

Functionality

• [x] Installation: Does installation proceed as outlined in the documentation?
• [x] Functionality: Have the functional claims of the software been confirmed?
• [x] Performance: If there are any performance claims of the software, have they been confirmed? (If there are no claims, please check off this item.)

Documentation

• [x] A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
• [x] Installation instructions: Is there a clearly-stated list of dependencies? Ideally these should be handled with an automated package management solution.
• [x] Example usage: Do the authors include examples of how to use the software (ideally to solve real-world analysis problems).
• [x] Functionality documentation: Is the core functionality of the software documented to a satisfactory level (e.g., API method documentation)?
• [x] Automated tests: Are there automated tests or manual steps described so that the function of the software can be verified?
• [x] Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support

Software paper

• [x] Authors: Does the paper.md file include a list of authors with their affiliations?
• [x] A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
• [x] References: Do all archival references that should have a DOI list one (e.g., papers, datasets, software)?

[whedon]

Hello human, I'm @whedon, a robot that can help you with some common editorial tasks. @vsoch, it looks like you're currently assigned as the reviewer for this paper :tada:.

:star: Important :star:

If you haven't already, you should seriously consider unsubscribing from GitHub notifications for this (https://github.com/openjournals/joss-reviews) repository. As a reviewer, you're probably currently watching this repository which means for GitHub's default behaviour you will receive notifications (emails) for all reviews 😿

To fix this do the following two things:

1. Set yourself as 'Not watching' https://github.com/openjournals/joss-reviews:

1. You may also like to change your default settings for this watching repositories in your GitHub profile here: https://github.com/settings/notifications

For a list of things I can do to help you, just type:

@whedon commands

[whedon]

Attempting PDF compilation. Reticulating splines etc...

[whedon]

:point_right: Check article proof :page_facing_up: :point_left:

[vsoch]

Note that I'm updating the current bitbucket repo in whedon's message to be https://github.com/gasparl/citapp_pc

[vsoch]

@whedon commands

[whedon]

Here are some things you can ask me to do:

# List Whedon's capabilities
@whedon commands

# List of editor GitHub usernames
@whedon list editors

# List of reviewers together with programming language preferences and domain expertise
@whedon list reviewers

# Compile the paper
@whedon generate pdf

[vsoch]

@whedon generate pdf

[whedon]

Attempting PDF compilation. Reticulating splines etc...

[whedon]

:point_right: Check article proof :page_facing_up: :point_left:

[gasparl]

@whedon generate pdf

[whedon]

Attempting PDF compilation. Reticulating splines etc...

[whedon]

:point_right: Check article proof :page_facing_up: :point_left:

[andytwoods]

Sorry for delays. Started a new job and things still settling down. A nice project. My minimal comments: https://github.com/gasparl/citapp_pc/issues/9

[gasparl]

@whedon generate pdf

[whedon]

Attempting PDF compilation. Reticulating splines etc...

[whedon]

:point_right: Check article proof :page_facing_up: :point_left:

[vsoch]

@yochannah - an update from me! @gasparl has responded to and fixed all of my issues, and done a fantastic job. Specifically, he has better written the manuscript for a general audience, and adding guidelines for contributing, getting help, and usage to the repository README, along with adding a demo depoyment to Github pages.

The only box unchecked is with respect to automated tests. It looks like @andytwoods has made some suggestions for how to go about this in his review, so when @gasparl has completed any remaining issues to the satisfaction of @andytwoods I give my thumbs up as well. Bravo, overall great job @gasparl !

[gasparl]

Thanks very much @vsoch for your really kind and useful review!

(Yes, the testing function is in progress.)

[yochannah]

Looks like we're moving forward nicely! @vsoch, @andytwoods, @gasparl - let me know when we're happy with everything and ready to move forwards.

[vsoch]

I'm really interested to learn about the CI strategy that is underway to test the web interface. I've done some testing with selenium (via python) but I don't think that's best practice, so I'm looking forward to learning the details. :)

[vsoch]

@gasparl has gone above and beyond to add automated testing for his application. I have checked this off as the final box to finish my review, and I'm ready to move forward with acceptance of the work for publication in JOSS.

[gasparl]

@whedon generate pdf

[whedon]

Attempting PDF compilation. Reticulating splines etc...

[whedon]

:point_right: Check article proof :page_facing_up: :point_left:

[yochannah]

Wow, the work y'all have put into this is amazing! 💯 @andytwoods what are your thoughts at the minute - are we ready to check off the last few boxes?

[yochannah]

Okay, I've run through the paper again and it all looks good - links in the references work and I didn't spot any typos in need of correction ✅

@gasparl - At this point could you make an archive of the reviewed software in Zenodo/figshare/other service and update this thread with the DOI of the archive? It's probably worth giving the manuscript a final proofread if you feel you need to, as well.

[gasparl]

@whedon set 10.5281/zenodo.2561206 as archive

[whedon]

I'm sorry @gasparl, I'm afraid I can't do that. That's something only editors are allowed to do.

[gasparl]

I'm sorry @gasparl, I'm afraid I can't do that. That's something only editors are allowed to do.

Well :)
Anyway, @yochannah, you see the DOI: 10.5281/zenodo.2561206

[gasparl]

By the way, the version is now 1.0.3, not 1.0.0 (as on this page). Also, the Repository Address (and name) of the submitted paper (http://joss.theoj.org/papers/b2d9ee93da00cb5b3fdc962d50df5063) is still the old bitbucket link. I don't think I can edit either myself.

[yochannah]

@whedon set 1.0.3 as version

[whedon]

OK. 1.0.3 is the version.

[yochannah]

@whedon set 10.5281/zenodo.2561206 as archive

[whedon]

OK. 10.5281/zenodo.2561206 is the archive.

[yochannah]

@whedon generate pdf

[whedon]

Attempting PDF compilation. Reticulating splines etc...

[whedon]

:point_right: Check article proof :page_facing_up: :point_left:

[yochannah]

@gasparl

1. I've updated the version to be 1.0.3 - Thanks!
2. The proof above has the newer GitHub repo linked, and I noticed that one of the reviewers links still has BitBucket so I replaced it with the github link. If this all looks okay to you (I think it should be fine) I'll ping the editors-in-chief team and we'll accept the paper! 🎉

[gasparl]

Super. It all looks perfectly fine to me, I also proofread the paper one last time. Thanks very much @yochannah!

[gasparl]

Though, just to be sure, regarding the old repository address, what I meant is that at http://joss.theoj.org/papers/active?page=7, I see this:

And then also when I click on it:

But then perhaps this will automatically change when accepted?

[yochannah]

I see what you mean! That's a good question I'm not sure about the answer to.

@whedon commands

[yochannah]

@whedon commands

[whedon]

Here are some things you can ask me to do:

# List all of Whedon's capabilities
@whedon commands

# Assign a GitHub user as the sole reviewer of this submission
@whedon assign @username as reviewer

# Add a GitHub user to the reviewers of this submission
@whedon add @username as reviewer

# Remove a GitHub user from the reviewers of this submission
@whedon remove @username as reviewer

# List of editor GitHub usernames
@whedon list editors

# List of reviewers together with programming language preferences and domain expertise
@whedon list reviewers

# Change editorial assignment
@whedon assign @username as editor

# Set the software archive DOI at the top of the issue e.g.
@whedon set 10.0000/zenodo.00000 as archive

# Set the software version at the top of the issue e.g.
@whedon set v1.0.1 as version

# Open the review issue
@whedon start review

EDITORIAL TASKS

# Compile the paper
@whedon generate pdf

# Ask Whedon to accept the paper and deposit with Crossref
@whedon accept

# Ask Whedon to check the references for missing DOIs
@whedon check references

[yochannah]

hmmm, there's definitely nothing in the whedon commands listed there to update the repo URL.

@openjournals/joss-eics - Can you help? Somewhere some metadata still has the old repo pointing to it though - see this comment from @gasparl above. I've modified any links in the pre-review and review issues to the correct URL, but maybe I missed something?

• correct repo: https://github.com/gasparl/citapp_pc
• wrong repo: https://bitbucket.org/gaspar_lukacs/citapp_pc

Other than that, I think we're ready to accept this paper!! 🎉 💯 🏆 Thank you so much @vsoch and @andytwoods for your reviews and @gasparl for working with the reviewers to get this repo ready for publication.

[arfon]

@openjournals/joss-eics - Can you help? Somewhere some metadata still has the old repo pointing to it though - see this comment from @gasparl above. I've modified any links in the pre-review and review issues to the correct URL, but maybe I missed something?

Sorry, this is a manual process to update on the JOSS site. I've done this now.

[arfon]

@whedon accept

[whedon]

Attempting dry run of processing paper acceptance...

[whedon]

PDF failed to compile for issue #1179 with the following error:

% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed

0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0
100 15 0 15 0 0 174 0 --:--:-- --:--:-- --:--:-- 174
sh: 0: getcwd() failed: No such file or directory
pandoc: 10.21105.joss.01179.pdf: openBinaryFile: does not exist (No such file or directory)
Looks like we failed to compile the PDF

[whedon]

```Reference check summary:

OK DOIs

• http://doi.org/10.1016/j.jarmac.2017.01.013 is OK
• http://doi.org/10.1037//0021-9010.85.1.30 is OK
• http://doi.org/10.1037/a0023589 is OK

MISSING DOIs

• https://doi.org/10.1016/j.actpsy.2019.01.010 may be missing for title: Information leakage in the response time-based Concealed Information Test. Manuscript submitted

INVALID DOIs

• None
  ```

[arfon]

@whedon accept

[whedon]

Attempting dry run of processing paper acceptance...

[vsoch]

whedon, stop being a bad robot!

[whedon]

```Reference check summary:

OK DOIs

• http://doi.org/10.1016/j.jarmac.2017.01.013 is OK
• http://doi.org/10.1037//0021-9010.85.1.30 is OK
• http://doi.org/10.1037/a0023589 is OK

MISSING DOIs

• https://doi.org/10.1016/j.actpsy.2019.01.010 may be missing for title: Information leakage in the response time-based Concealed Information Test. Manuscript submitted

INVALID DOIs

• None
  ```

[arfon]

Not sure what's up with @whedon here. I've compiled and accepted the paper locally.

@vsoch @andytwoods - many thanks for your reviews here and to @yochannah for editing this submission ✨

@gasparl - your paper is now accepted into JOSS :zap::rocket::boom:

[whedon]

:tada::tada::tada: Congratulations on your paper acceptance! :tada::tada::tada:

If you would like to include a link to your paper from your README use the following code snippets:

Markdown:
[![DOI](http://joss.theoj.org/papers/10.21105/joss.01179/status.svg)](https://doi.org/10.21105/joss.01179)

HTML:
<a style="border-width:0" href="https://doi.org/10.21105/joss.01179">
  <img src="http://joss.theoj.org/papers/10.21105/joss.01179/status.svg" alt="DOI badge" >
</a>

reStructuredText:
.. image:: http://joss.theoj.org/papers/10.21105/joss.01179/status.svg
   :target: https://doi.org/10.21105/joss.01179

This is how it will look in your documentation:

We need your help!

Journal of Open Source Software is a community-run journal and relies upon volunteer effort. If you'd like to support us please consider doing either one (or both) of the the following:

• Volunteering to review for us sometime in the future. You can add your name to the reviewer list here: http://joss.theoj.org/reviewer-signup.html
• Making a small donation to support our running costs here: https://www.flipcause.com/secure/cause_pdetails/Mjk3Nzk=

[gasparl]

Thanks very much @arfon (and once again @yochannah) for correcting this!

This is an awesome review process, definitely not my last submission here.

[whedon]

Check final proof :point_right: https://github.com/openjournals/joss-papers/pull/508

If the paper PDF and Crossref deposit XML look good in https://github.com/openjournals/joss-papers/pull/508, then you can now move forward with accepting the submission by compiling again with the flag deposit=true e.g.
@whedon accept deposit=true