Joss-reviews: [REVIEW]: View eXtract aNnotate (ViXeN) media files: a multimedia project manager

[whedon]

Submitting author: @prabhuramachandran (Prabhu Ramachandran)
Repository: https://github.com/vixen-project/vixen
Version: 1.0rc3
Editor: @xuanxu
Reviewer: @deniederhut, @amoeba
Archive: 10.5281/zenodo.3464327

Status

Status badge code:

HTML: <a href="http://joss.theoj.org/papers/57a00170ef85a6b3bb8190e3e030fbdb"><img src="http://joss.theoj.org/papers/57a00170ef85a6b3bb8190e3e030fbdb/status.svg"></a>
Markdown: [![status](http://joss.theoj.org/papers/57a00170ef85a6b3bb8190e3e030fbdb/status.svg)](http://joss.theoj.org/papers/57a00170ef85a6b3bb8190e3e030fbdb)

Reviewers and authors:

Please avoid lengthy details of difficulties in the review thread. Instead, please create a new issue in the target repository and link to those issues (especially acceptance-blockers) by leaving comments in the review thread below. (For completists: if the target issue tracker is also on GitHub, linking the review thread in the issue or vice versa will create corresponding breadcrumb trails in the link target.)

Reviewer instructions & questions

@deniederhut & @amoeba, please carry out your review in this issue by updating the checklist below. If you cannot edit the checklist please:

1. Make sure you're logged in to your GitHub account
2. Be sure to accept the invite at this URL: https://github.com/openjournals/joss-reviews/invitations

The reviewer guidelines are available here: https://joss.readthedocs.io/en/latest/reviewer_guidelines.html. Any questions/concerns please let @xuanxu know.

✨ Please try and complete your review in the next two weeks ✨

Review checklist for @deniederhut

Conflict of interest

• [x] As the reviewer I confirm that I have read the JOSS conflict of interest policy and that there are no conflicts of interest for me to review this work.

Code of Conduct

• [x] I confirm that I read and will adhere to the JOSS code of conduct.

General checks

• [x] Repository: Is the source code for this software available at the repository url?
• [x] License: Does the repository contain a plain-text LICENSE file with the contents of an OSI approved software license?
• [x] Version: 1.0rc3
• [x] Authorship: Has the submitting author (@prabhuramachandran) made major contributions to the software? Does the full list of paper authors seem appropriate and complete?

Functionality

• [x] Installation: Does installation proceed as outlined in the documentation?
• [x] Functionality: Have the functional claims of the software been confirmed?
• [x] Performance: If there are any performance claims of the software, have they been confirmed? (If there are no claims, please check off this item.)

Documentation

• [x] A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
• [x] Installation instructions: Is there a clearly-stated list of dependencies? Ideally these should be handled with an automated package management solution.
• [x] Example usage: Do the authors include examples of how to use the software (ideally to solve real-world analysis problems).
• [x] Functionality documentation: Is the core functionality of the software documented to a satisfactory level (e.g., API method documentation)?
• [x] Automated tests: Are there automated tests or manual steps described so that the function of the software can be verified?
• [x] Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support

Software paper

• [x] Authors: Does the paper.md file include a list of authors with their affiliations?
• [x] A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
• [x] References: Do all archival references that should have a DOI list one (e.g., papers, datasets, software)?

Review checklist for @amoeba

Conflict of interest

• [x] As the reviewer I confirm that I have read the JOSS conflict of interest policy and that there are no conflicts of interest for me to review this work.

Code of Conduct

• [x] I confirm that I read and will adhere to the JOSS code of conduct.

General checks

• [x] Repository: Is the source code for this software available at the repository url?
• [x] License: Does the repository contain a plain-text LICENSE file with the contents of an OSI approved software license?
• [x] Version: 1.0rc3
• [x] Authorship: Has the submitting author (@prabhuramachandran) made major contributions to the software? Does the full list of paper authors seem appropriate and complete?

Functionality

• [x] Installation: Does installation proceed as outlined in the documentation?
• [x] Functionality: Have the functional claims of the software been confirmed?
• [x] Performance: If there are any performance claims of the software, have they been confirmed? (If there are no claims, please check off this item.)

Documentation

• [x] A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
• [x] Installation instructions: Is there a clearly-stated list of dependencies? Ideally these should be handled with an automated package management solution.
• [x] Example usage: Do the authors include examples of how to use the software (ideally to solve real-world analysis problems).
• [x] Functionality documentation: Is the core functionality of the software documented to a satisfactory level (e.g., API method documentation)?
• [x] Automated tests: Are there automated tests or manual steps described so that the function of the software can be verified?
• [x] Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support

Software paper

• [x] Authors: Does the paper.md file include a list of authors with their affiliations?
• [x] A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
• [x] References: Do all archival references that should have a DOI list one (e.g., papers, datasets, software)?

[whedon]

Hello human, I'm @whedon, a robot that can help you with some common editorial tasks. @deniederhut, @amoeba it looks like you're currently assigned to review this paper :tada:.

:star: Important :star:

If you haven't already, you should seriously consider unsubscribing from GitHub notifications for this (https://github.com/openjournals/joss-reviews) repository. As a reviewer, you're probably currently watching this repository which means for GitHub's default behaviour you will receive notifications (emails) for all reviews 😿

To fix this do the following two things:

1. Set yourself as 'Not watching' https://github.com/openjournals/joss-reviews:

1. You may also like to change your default settings for this watching repositories in your GitHub profile here: https://github.com/settings/notifications

For a list of things I can do to help you, just type:

@whedon commands

[whedon]

Attempting PDF compilation. Reticulating splines etc...

[whedon]

:point_right: Check article proof :page_facing_up: :point_left:

[arfon]

@deniederhut, @amoeba - please carry out your review in this issue by updating the checklist above and giving feedback in this issue. The reviewer guidelines are available here: https://joss.readthedocs.io/en/latest/reviewer_guidelines.html

Any questions/concerns please let @xuanxu know.

[amoeba]

I'm almost done and had one review checklist item I couldn't check off: https://github.com/vixen-project/vixen/issues/23. This review is a Conditional Accept for now.

@prabhuramachandran, can you please take a look at https://github.com/vixen-project/vixen/issues/23 and let me know what you think? Overall, ViXeN looks and works great. You've clearly put a lot of time and effort into building it and testing it. I found the installation as easy as you outlined, running it to be straight-forward, and creating a test project for tagging some screenshots I had laying around was easy (esp. with your docs). The documentation was well-written and was useful as I went. The ability to add tags of various data types and add Processors is really nice. I can see myself recommending this to colleagues. As a bonus, I was happy to see what looked like a decent test suite covering the various components of the application. I sent in a PR for some minor typos but otherwise only had one issue which I've described in https://github.com/vixen-project/vixen/issues/23. Please take a look!

[amoeba]

Hey @xuanxu, @prabhuramachandran has resolved my outstanding issue and my review is now an Accept.

[xuanxu]

Great, thanks @amoeba!

[xuanxu]

@deniederhut: did you had the opportunity to start your review?

[deniederhut]

Nope. We're in the middle of the reviews for SciPy.

On Tue, Jun 25, 2019, 10:47 Juanjo Bazán [e-mail] wrote:

@deniederhut https://github.com/deniederhut: did you had the
opportunity to start your review?

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/openjournals/joss-reviews/issues/1501?email_source=notifications&email_token=ABUM3LBI2UTG6RBUDZT7GIDP4I4ZRA5CNFSM4HXG6QJ2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODYQWH2I#issuecomment-505504745,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ABUM3LDFDGEZMZRMCZEHSEDP4I4ZRANCNFSM4HXG6QJQ
.

[xuanxu]

Hey @deniederhut, do you have an estimation on when will you be available for the review? If so I can set a reminder via whedon.

[deniederhut]

Oh how thoughtful! Yeah, I should have time next week.

On Wed, Jul 10, 2019, 02:57 Juanjo Bazán [e-mail] wrote:

Hey @deniederhut https://github.com/deniederhut, do you have an
estimation on when will you be available for the review? If so I can set a
reminder via whedon.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/openjournals/joss-reviews/issues/1501?email_source=notifications&email_token=ABUM3LB7Y556DUAJPVIELILP6WI7VA5CNFSM4HXG6QJ2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODZSUNXI#issuecomment-509953757,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ABUM3LDGTAANY2XKF3FZZGLP6WI7VANCNFSM4HXG6QJQ
.

[xuanxu]

@whedon remind @deniederhut in five days

[whedon]

Reminder set for @deniederhut in five days

[whedon]

:wave: @deniederhut, please update us on how your review is going.

[deniederhut]

👋 @whedon it's on my to-do list for tomorrow

On Tue, Jul 16, 2019, 10:20 whedon [e-mail] wrote:

👋 @deniederhut https://github.com/deniederhut, please update us on how
your review is going.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/openjournals/joss-reviews/issues/1501?email_source=notifications&email_token=ABUM3LG7ZOM6HXYMULS26ALP7XRNTA5CNFSM4HXG6QJ2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD2BGSEY#issuecomment-511863059,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ABUM3LHADD573ML5DACRZ5DP7XRNTANCNFSM4HXG6QJQ
.

[deniederhut]

Hey @xuanxu ! I have a COI question. In the last year, Prabhu and I worked for the same company (Enthought). We were not working on the same project, and at no time was one of us a report of the other. Can I still review this submission?

[xuanxu]

@deniederhut Thanks for noting it! If you are not currently affiliated with the same institution/company and you didn’t work on the same project then by the current policy I see at most a minor COI, that can be waived if you think you are able to make an impartial assessment of this submission.

[deniederhut]

Okay cool. I like Prabhu a lot, but I think I can still be sufficiently impartial.

I see that I am supposed to leave comments about the codebase on its own issue tracker - where do I put comments about the paper?

[xuanxu]

@deniederhut This is the main issue where the review takes place. You can add comments about the paper here. If you also create issues or pull requests in the submitted software’s issue tracker, please mention/link them here.

[xuanxu]

:wave: @prabhuramachandran: Any progress with the pending issues?

[prabhuramachandran]

Hi @xuanxu -- the syntax error is something I will fix by setting a python_requires this weekend as fixing the dependent package to support 3.7 opens a can of worms that requires much more time for me to address right now (the issue isn't vixen but the jigna package which doesn't support Python 3.7).

Issue 29 is something I am having a hard time reproducing so will try and spend some time on that but I have an alternative that does work, which is to use the binaries. If it is OK to complete the reviews with the binaries, that may be more efficient.

[xuanxu]

@prabhuramachandran great, as long as all that is documented, i.e. readme states clearly the required Python, the binaries installation steps, etc, I think is ok.

[Kevin-Mattheus-Moerman]

@deniederhut thanks again for your help here! Can you give an update on review progress?

[deniederhut]

Sure thing! I think we're currently waiting on a fix for the python version
requirement.

On Wed, Sep 11, 2019, 05:25 Kevin Mattheus Moerman [e-mail]
wrote:

@deniederhut https://github.com/deniederhut thanks again for your help
here! Can you give an update on review progress?

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/openjournals/joss-reviews/issues/1501?email_source=notifications&email_token=ABUM3LFECROJOL2E3CEQ573QJDBSXA5CNFSM4HXG6QJ2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD6OAYUQ#issuecomment-530320466,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ABUM3LAUKJ7H5R54EGSNDSTQJDBSXANCNFSM4HXG6QJQ
.

[prabhuramachandran]

I've just merge a tiny fix for that. I think you should be able to test the software itself with the binaries available. I will resolve the other issues as time permits.

[deniederhut]

Thanks Prabhu! I've confirmed that the python requires works as expected.

@xuanxu this looks good to me!

[prabhuramachandran]

@deniederhut -- thank you and sorry I took so long to get to fixing the issue!

[xuanxu]

Great, thanks @deniederhut & @prabhuramachandran!

[xuanxu]

@whedon generate pdf

[whedon]

Attempting PDF compilation. Reticulating splines etc...

[whedon]

:point_right: Check article proof :page_facing_up: :point_left:

[xuanxu]

@whedon check references

[whedon]

Attempting to check references...

[whedon]

```Reference check summary:

OK DOIs

• 10.1111/2041-210X.12892 is OK

MISSING DOIs

• None

INVALID DOIs

• None
  ```

[xuanxu]

Ok, everything looks good @prabhuramachandran, here are the next steps:

• Please make a new tagged release that includes all changes made during this review
• Archive that release in Zenodo
• Check the Zenodo deposit has the correct metadata: title and author name should match the paper; you may also add your ORCID.

Once you do that please report here the version number and the Zenodo DOI.

[xuanxu]

@prabhuramachandran: friendly ping :) Looks like we are just waiting for your archive.

[prabhuramachandran]

Yes indeed, I am sorry, I will do this over the coming weekend -- this month has been a very tight.

[prabhuramachandran]

@xuanxu -- Thanks for the reminder. I have uploaded version 1.0rc3 on pypi and also on zenodo here: http://doi.org/10.5281/zenodo.3464327 . This is my first zenodo upload so I hope I did not mess anything up. I will now make the binaries for people to use and upload them on the vixen github releases.

[xuanxu]

@whedon set 1.0rc3 as version

[whedon]

OK. 1.0rc3 is the version.

[xuanxu]

@whedon set 10.5281/zenodo.3464327 as archive

[whedon]

OK. 10.5281/zenodo.3464327 is the archive.

[xuanxu]

All ready for publication.
Pinging @openjournals/joss-eics for final acceptance.

[kyleniemeyer]

@whedon check references

[whedon]

Attempting to check references...

[whedon]

```Reference check summary:

OK DOIs

• 10.1111/2041-210X.12892 is OK

MISSING DOIs

• None

INVALID DOIs

• None
  ```

[kyleniemeyer]

Hi @prabhuramachandran, while going through the final checks I noticed the (sole) citation to a prior paper from last year, also about ViXeN.

How is this submission reporting on different scholarship than the article "ViXeN: An open‐source package for managing multimedia data", published in _Methods in Ecology and Evolution_ ? They both appear to be software papers about the same software. While we do encourage authors to submit a software submission to JOSS and research articles using the software to other journals, the 2018 article seems to be just about the software.

[prabhuramachandran]

@kyleniemeyer -- Thanks for checking! The Methods in Ecology and Evolution paper was really about the application in the domain of ecology and camera trap videos/images and looked at a case study from the perspective of field biologists. It talks about other tools relevant to the area and how this tool is useful in that context. The JOSS paper doesn't focus on the ecological application but on the fact that this is a general purpose multimedia annotation tool which is broader in scope. We were given to understand that the primary purpose in JOSS is to review the software and not as an application in a particular domain of interest. I am not sure if this answers the question. Am happy to provide any clarifications. Thanks.

[kyleniemeyer]

@prabhuramachandran You're right that our purpose here is to review the software only, and not results—in fact, we do not really allow inclusion of results in our article (only examples). However, the concern here is that the other paper already describes the software in detail, in effect providing a "paper credit" for the software development, and a JOSS article would repeat that.

Has the scope or functionality of the software changed since it was described in the prior paper?

[prabhuramachandran]

@kyleniemeyer -- I am sorry if I completely misunderstood the scope of JOSS papers and wasted everyone's time. While we discussed the software in the previous paper, the review process did not go into checking the software the way JOSS does or concern itself with archiving it etc. When we saw the nice talks at SciPy on JOSS, we thought we might submit it here to check it out. At the time, I recall reading an FAQ that said that it was OK to submit the software here despite there being another paper that discusses the software in a more traditional context. I assumed that given the different focus of the two, it was acceptable. I am terribly sorry if I misunderstood. It was totally not my intention to waste everyone's time (or get extra credit). I leave it to you to decide about this submission as you see fit and will defer to your decision on this. Apologies again.

[kyleniemeyer]

@prabhuramachandran totally understand—we don't have entirely clear guidelines on this, as it hasn't come up too often. We'll discuss internally and get back to you. Either way, it looks like the software is great and the review process has made it even better :)

[prabhuramachandran]

@kyleniemeyer sure, thank you! Yes, the review process is very good indeed and we are very grateful for their time, attention, and feedback.

[kyleniemeyer]

Hi @prabhuramachandran, the Associate Editors in Chief discussed this issue, and unfortunately we decided that we cannot publish this article, because we consider the prior article a software paper analogous to—albeit longer than—the JOSS article (even if the review process did not focus on the software like ours).

We do completely support JOSS authors in submitting research articles that use their published software, where the separate articles are about the results using the software and not the software itself. But that doesn't appear to be the case here.

I'm sorry for the confusion about this—we will work to improve our documentation to clarify our policy in such cases.

Regardless, we appreciate the efforts of @deniederhut and @amoeba in giving feedback on the software, and @xuanxu in handling this.