Joss-reviews: [REVIEW]: DynaMo: Dynamic Body Shape and Motion Capture with Intel RealSense Cameras

[whedon]

Submitting author: @abhishektha (Abhishektha Boppana)
Repository: https://github.com/anderson-cu-bioastronautics/dynamo_realsense-capture
Version: v1.1
Editor: @labarba
Reviewers: @melund, @ixjlyons, @alcantarar
Archive: 10.5281/zenodo.3464497

Status

Status badge code:

HTML: <a href="http://joss.theoj.org/papers/7af4e58d74943cd41832d1320c83a897"><img src="http://joss.theoj.org/papers/7af4e58d74943cd41832d1320c83a897/status.svg"></a>
Markdown: [![status](http://joss.theoj.org/papers/7af4e58d74943cd41832d1320c83a897/status.svg)](http://joss.theoj.org/papers/7af4e58d74943cd41832d1320c83a897)

Reviewers and authors:

Please avoid lengthy details of difficulties in the review thread. Instead, please create a new issue in the target repository and link to those issues (especially acceptance-blockers) by leaving comments in the review thread below. (For completists: if the target issue tracker is also on GitHub, linking the review thread in the issue or vice versa will create corresponding breadcrumb trails in the link target.)

Reviewer instructions & questions

@melund & @ixjlyons, please carry out your review in this issue by updating the checklist below. If you cannot edit the checklist please:

1. Make sure you're logged in to your GitHub account
2. Be sure to accept the invite at this URL: https://github.com/openjournals/joss-reviews/invitations

The reviewer guidelines are available here: https://joss.readthedocs.io/en/latest/reviewer_guidelines.html. Any questions/concerns please let @labarba know.

✨ Please try and complete your review in the next two weeks ✨

Review checklist for @melund

Conflict of interest

• [x] As the reviewer I confirm that I have read the JOSS conflict of interest policy and that there are no conflicts of interest for me to review this work.

Code of Conduct

• [x] I confirm that I read and will adhere to the JOSS code of conduct.

General checks

• [x] Repository: Is the source code for this software available at the repository url?
• [x] License: Does the repository contain a plain-text LICENSE file with the contents of an OSI approved software license?
• [x] Version: v1.1
• [x] Authorship: Has the submitting author (@abhishektha) made major contributions to the software? Does the full list of paper authors seem appropriate and complete?

Functionality

• [x] Installation: Does installation proceed as outlined in the documentation?
• [ ] Functionality: Have the functional claims of the software been confirmed?
• [x] Performance: If there are any performance claims of the software, have they been confirmed? (If there are no claims, please check off this item.)

Documentation

• [x] A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
• [x] Installation instructions: Is there a clearly-stated list of dependencies? Ideally these should be handled with an automated package management solution.
• [x] Example usage: Do the authors include examples of how to use the software (ideally to solve real-world analysis problems).
• [x] Functionality documentation: Is the core functionality of the software documented to a satisfactory level (e.g., API method documentation)?
• [x] Automated tests: Are there automated tests or manual steps described so that the function of the software can be verified?
• [x] Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support

Software paper

• [x] Authors: Does the paper.md file include a list of authors with their affiliations?
• [x] A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
• [x] References: Do all archival references that should have a DOI list one (e.g., papers, datasets, software)?

Review checklist for @ixjlyons

Conflict of interest

• [x] As the reviewer I confirm that I have read the JOSS conflict of interest policy and that there are no conflicts of interest for me to review this work.

Code of Conduct

• [x] I confirm that I read and will adhere to the JOSS code of conduct.

General checks

• [x] Repository: Is the source code for this software available at the repository url?
• [x] License: Does the repository contain a plain-text LICENSE file with the contents of an OSI approved software license?
• [x] Version: v1.1
• [x] Authorship: Has the submitting author (@abhishektha) made major contributions to the software? Does the full list of paper authors seem appropriate and complete?

Functionality

• [ ] Installation: Does installation proceed as outlined in the documentation?
• [ ] Functionality: Have the functional claims of the software been confirmed?
• [ ] Performance: If there are any performance claims of the software, have they been confirmed? (If there are no claims, please check off this item.)

Documentation

• [x] A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
• [x] Installation instructions: Is there a clearly-stated list of dependencies? Ideally these should be handled with an automated package management solution.
• [x] Example usage: Do the authors include examples of how to use the software (ideally to solve real-world analysis problems).
• [x] Functionality documentation: Is the core functionality of the software documented to a satisfactory level (e.g., API method documentation)?
• [x] Automated tests: Are there automated tests or manual steps described so that the function of the software can be verified?
• [x] Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support

Software paper

• [x] Authors: Does the paper.md file include a list of authors with their affiliations?
• [x] A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
• [x] References: Do all archival references that should have a DOI list one (e.g., papers, datasets, software)?

Review checklist for @alcantarar

Conflict of interest

• [x] As the reviewer I confirm that I have read the JOSS conflict of interest policy and that there are no conflicts of interest for me to review this work.

Code of Conduct

• [x] I confirm that I read and will adhere to the JOSS code of conduct.

General checks

• [x] Repository: Is the source code for this software available at the repository url?
• [ ] License: Does the repository contain a plain-text LICENSE file with the contents of an OSI approved software license?
• [ ] Version: v1.1
• [x] Authorship: Has the submitting author (@abhishektha) made major contributions to the software? Does the full list of paper authors seem appropriate and complete?

Functionality

• [x] Installation: Does installation proceed as outlined in the documentation?
• [x] Functionality: Have the functional claims of the software been confirmed?
• [x] Performance: If there are any performance claims of the software, have they been confirmed? (If there are no claims, please check off this item.)

Documentation

• [ ] A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
• [ ] Installation instructions: Is there a clearly-stated list of dependencies? Ideally these should be handled with an automated package management solution.
• [ ] Example usage: Do the authors include examples of how to use the software (ideally to solve real-world analysis problems).
• [ ] Functionality documentation: Is the core functionality of the software documented to a satisfactory level (e.g., API method documentation)?
• [ ] Automated tests: Are there automated tests or manual steps described so that the function of the software can be verified?
• [ ] Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support

Software paper

• [ ] Authors: Does the paper.md file include a list of authors with their affiliations?
• [ ] A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
• [ ] References: Do all archival references that should have a DOI list one (e.g., papers, datasets, software)?

[whedon]

Hello human, I'm @whedon, a robot that can help you with some common editorial tasks. @melund, it looks like you're currently assigned as the reviewer for this paper :tada:.

:star: Important :star:

If you haven't already, you should seriously consider unsubscribing from GitHub notifications for this (https://github.com/openjournals/joss-reviews) repository. As a reviewer, you're probably currently watching this repository which means for GitHub's default behaviour you will receive notifications (emails) for all reviews 😿

To fix this do the following two things:

1. Set yourself as 'Not watching' https://github.com/openjournals/joss-reviews:

1. You may also like to change your default settings for this watching repositories in your GitHub profile here: https://github.com/settings/notifications

For a list of things I can do to help you, just type:

@whedon commands

[whedon]

Attempting PDF compilation. Reticulating splines etc...

[whedon]

:point_right: Check article proof :page_facing_up: :point_left:

[labarba]

👋 @melund, @ixjlyons — Thank you for agreeing to review for JOSS! This is where the action happens: work your way through the review checklist, feel free to ask questions or post comments here, and also open issues in the submission repository as needed. Godspeed!

[labarba]

@ixjlyons — I see that you haven't checked off any items in your review checklist. Have you been able to get a start on this? Let me know if you have any questions!

[ixjlyons]

Sorry I haven't gotten to this yet. I will try to get started this week and complete a review over the weekend.

[labarba]

Hi @ixjlyons — I think you meant to work on this review over the past weekend. Can you give us a status update? We can also set an automatic reminder, if you have a new ETA.

[ixjlyons]

Apologies for the delay. Here is my review:

Paper

The paper is written clearly and offers insight into the functionality of the library. A few minor points:

• Unresolved reference in first paragraph.
• Figure 1 not showing.
• 2nd from last paragraph: "synchroniously" -> "synchronously"
• I think the last paragraph describing some research applications could be better integrated into the background section of the paper.

Docs

The documentation is distributed into the README, a couple other markdown documents, and a Jupyter notebook. These work together to cover much of the library's functionality and usage, but you could benefit from a dedicated documentation site with API documentation generated from the docstrings. Every function appears to have docstrings and those files are browsable through GitHub, so I will defer to @labarba on whether or not that satisfies the functionality documentation check.

A few other minor issues:

• Links to files are broken in this section: https://github.com/anderson-cu-bioastronautics/dynamo_realsense-capture/tree/b3f688389712a5be60f373a3fbefa45542519f01#included-functions
• You may want to mention that the kabsch function in the tutorial notebook is a function that's available from the library and not something the user has to define.
• The image at the bottom of the tutorial notebook is missing.

Code/Functionality

Unfortunately, I wasn't able to do much with Dynamo being stuck with Linux (no pyrealsense2) and lacking D4XX cameras, so I'm leaving the installation, functionality, and performance checks un-checked.

I notice that some of the modules (e.g. calculate_rmsd.py) can be run as scripts. Consider packaging that functionality as console scripts, or perhaps mention that they can be run this way in the documentation.

Just out of curiosity, have you considered additional storage formats aside from pickle? HDF5 or other formats might work well and could provide better portability.

Other

The tests are not automated, but as far as I know the instructions for running the tests manually suffices for the "automated tests" check. This library poses some challenges for automated testing (with dependence on hardware), though some of the computational aspects of the software could be automatically tested.

Overall

Overall I think Dynamo would benefit from a dedicated documentation page with API documentation and perhaps automated testing of functionality that doesn't depend on presence of specific hardware. The documentation seems fairly complete, but a more cohesive presentation of it could help newcomers to the library.

[melund]

@labarba. I am also done with my review. Specific feedback has been handled on the issue tracker:

Since I haven't got access to the cameras I couldn't review all part of the functionality. But what I could check and review does comply with the requirements of JOSS.

I agree with @ixjlyons about the lack of API documentation, but it is not blocking in my view. I think the most important comment is the lack of a conda-forge package. It would help guarantee that this package continues to work in the future. However, the current state is in accordance with the requirements of JOSS.

[abhishektha]

@whedon generate pdf

[whedon]

Attempting PDF compilation. Reticulating splines etc...

[whedon]

:point_right: Check article proof :page_facing_up: :point_left:

[abhishektha]

Hi all,

Thank you for the insightful comments and help with getting DynaMo up to JOSS standards so far. I've fixed the errors in the paper and tutorial Jupyter Notebook.

Is there anything else needed from me? I am a little unclear if I need to add to the documentation to proceed with the paper. Would a new section on the github which shows how to call each function of the package meet this requirement? If so I'm happy to get that done soon.

I will also work on getting this package onto conda-forge. It took me a while to catch up on reading how conda-forge recipes work but I think once we have pyrealsense2 on conda-forge, I can easily create a package for DynaMo.

Thanks again for all you contributions so far!

[melund]

It took me a while to catch up on reading how conda-forge recipes work but I think once we have pyrealsense2 on conda-forge, I can easily create a package for DynaMo.

That is great, but I think you will have to initiate the work with a pyrealsense2 conda-forge package. I think a win only package would be Ok for the first iteration. If you start the work, there is a good chance others will chip in and help.

[abhishektha]

@labarba Is there anything specific that I need to do for us to proceed with the review? I believe I addressed all the reviewer's comments (please let me know if I didn't). I am a bit confused from the reviews if I need to improve the documentation to meet the JOSS standards (since @melund mentioned it was non-blocking), so please let me know what is needed from me.

[labarba]

Hi @abhishektha — neither of the two reviewers have been able to check the functionality of the software, due to the hardware dependency (cameras) and lack of access. Accepting the paper under those conditions is awkward enough, so I would request that you make every effort at improvements that you can, given the reviewer comments.

Since you already have docstrings throughout, it should not be too hard to deploy API documentation using Sphinx? Please have a look at that and let me know what you think.

In the meantime, would you be able to suggest someone who may have access to the cameras to provide a check of functionality through a partial & supplementary review?

[abhishektha]

Thanks for the reply @labarba! I can definitely deploy API documentation through Sphinx, and will let you and the reviewers know when that is done.

Would it be okay if we had someone independently review functionality of the code using our own camera setup in our lab? We have a few colleagues in the field who have not worked with us on this project but may be able to provide a check of functionality. We would simply provide them access to our set of cameras and allow them to hook up their own computer to check functionality.

[labarba]

hey @abhishektha ... do you have updates with regards to the documentation? As to your question, if you could get someone to contribute some functionality checks to this review, that would good, give the restrictions we face.

[abhishektha]

@labarba Sorry for the delay, I had some trouble using Sphinx with the numpydoc style of tooltips I had been using, but its all fixed now and we finally have API documentation hosted on Github Pages and linked from the readme. @ixjlyons, @melund, please let me know if this new API documentation meets the standards.

I would like to suggest @alcantarar to contribute functionality checks to the review. He has access to our cameras but has not worked on the development of this project.

[labarba]

👋 @alcantarar — would you be willing to contribute a partial review of this JOSS submission, given that the assigned reviewers both lack access to the needed hardware to confirm functionality?

[alcantarar]

I would be willing to assist with the review.
Should I just open issues in the Dynamo repo as needed and comment below when my review is complete? I was going to follow the functionality section of the review checklist you've provided above.

[labarba]

@whedon add @alcantarar as reviewer

[whedon]

OK, @alcantarar is now a reviewer

[labarba]

Hi @alcantarar — thanks for agreeing to provide an additional review. I have added a review checklist for you. If you don't want to check off all items, that is OK, but we do need the COI and CoC responses from you, and especially need the functionality items that the other two reviewers could not check. Godspeed!

[labarba]

👋 @alcantarar — Could we have a status update on your review? Thanks!

[alcantarar]

I've opened a few issues in the target repo that have been/are currently being addressed . I'll continue working on the most recent issue with @abhishektha this week.

[alcantarar]

hey @labarba my review is complete. There are a few open issues in the target repository about documentation, but @abhishektha can close them easily. I went all the way through the Example Tutorial and it functioned as advertised. Really fun using the cameras actually!

I've checked off the COI/COS and Functionality checklist above. Did I miss anything?

[abhishektha]

@labarba, i went through and checked the documentation issues and addressed and closed them. Please let me know if there's anything else you need from me!

[labarba]

@alcantarar — to clarify, you are only contributing a partial review, focusing on the functionality of the software, and do not plan to review the documentation and paper. Correct?

[alcantarar]

@labarba Correct. I only made suggestions about documentation that directly influenced my ability to test the functionality. These issues have been addressed.

[labarba]

@alcantarar — would you mind writing a brief report here about your review of the functionality, with the purpose of informing the other two reviewers, who checked other aspects of the submission? We will ask @melund and @ixjlyons to provide a recommendation of acceptance on the basis of your additional checks of functionality, so I think this would be helpful.

[alcantarar]

The target repository claims to allow users to capture object shape or reflective marker data using multiple Intel RealSense cameras. The hardware requirements and software dependencies in the README.md file are comprehensive and allowed me to use my laptop to test two Intel RealSense cameras. The conda environment also worked as intended in terms of installation.

Setting up the cameras was straightforward and required no additional software to install. Users will be able to plug in the cameras and immediately be able to execute DynaMo functions to calibrate the cameras, stream/save images to their computer, and view 3D images afterwards. During my review, I ran into some issues with the number of parameters required for a given function or instances of hardcoding, but these issues have been addressed. Following the target repository's example page, I was able to record and save 3D object shape data from Intel RealSense Cameras, as the documentation claims.

[melund]

Thanks @alcantarar.

@labarba, I can recommend DynaMo for acceptance. Great work @abhishektha

[ixjlyons]

I also recommend accepting.

[abhishektha]

@labarba Is there anything you need from me to proceed with the paper?

[labarba]

@whedon generate pdf

[whedon]

Attempting PDF compilation. Reticulating splines etc...

[whedon]

:point_right: Check article proof :page_facing_up: :point_left:

[labarba]

Please see my copy editing of the paper on the linked PR.

[labarba]

@whedon check references

[whedon]

Attempting to check references...

[whedon]

```Reference check summary:

OK DOIs

• 10.1007/978-0-387-31439-6_472 is OK
• 10.1107/S0567739476001873 is OK

MISSING DOIs

• https://doi.org/10.1016/j.jbiomech.2008.06.024 may be missing for title: Systematic accuracy and precision analysis of video motion capturing systems–exemplified on the Vicon-460 system

INVALID DOIs

• https://doi.org/10.1016/j.displa.2013.08.011 is INVALID because of 'https://doi.org/' prefix
  ```

[labarba]

@abhishekbajpayee Can you now check if the DOIs flagged by whedon need fixing?

[abhishektha]

@whedon check references

[whedon]

Attempting to check references...

[labarba]

@whedon generate pdf

[whedon]

Attempting PDF compilation. Reticulating splines etc...

[whedon]

PDF failed to compile for issue #1466 with the following error:

Error reading bibliography ./paper.bib (line 30, column 1):
unexpected "u"
expecting space, ",", white space or "}"
Error running filter pandoc-citeproc:
Filter returned error status 1
Looks like we failed to compile the PDF

[abhishektha]

@whedon check references

[whedon]

Attempting to check references...

[abhishektha]

@whedon generate pdf

[whedon]

Attempting PDF compilation. Reticulating splines etc...

[whedon]

:point_right: Check article proof :page_facing_up: :point_left:

[whedon]

```Reference check summary:

OK DOIs

• 10.1016/j.displa.2013.08.011 is OK
• 10.1016/j.jbiomech.2008.06.024 is OK
• 10.1007/978-0-387-31439-6_472 is OK
• 10.1107/S0567739476001873 is OK

MISSING DOIs

• None

INVALID DOIs

• None
  ```

[abhishektha]

@whedon generate pdf

[whedon]

Attempting PDF compilation. Reticulating splines etc...

[whedon]

:point_right: Check article proof :page_facing_up: :point_left:

[abhishektha]

@whedon check references

[whedon]

Attempting to check references...

[whedon]

```Reference check summary:

OK DOIs

• 10.1016/j.displa.2013.08.011 is OK
• 10.1016/j.jbiomech.2008.06.024 is OK
• 10.1007/978-0-387-31439-6_472 is OK
• 10.1107/S0567739476001873 is OK

MISSING DOIs

• None

INVALID DOIs

• None
  ```

[abhishektha]

@labarba I think the DOIs are okay now. I did fix a missing reference for the OpenCV library, which was not provided with a DOI.

[whedon]

```Reference check summary:

OK DOIs

• 10.1016/j.displa.2013.08.011 is OK
• 10.1016/j.jbiomech.2008.06.024 is OK
• 10.1007/978-0-387-31439-6_472 is OK
• 10.1107/S0567739476001873 is OK

MISSING DOIs

• None

INVALID DOIs

• None
  ```

[labarba]

@whedon accept

[whedon]

No archive DOI set. Exiting...

[labarba]

@abhishekbajpayee — We need the following pre-publication steps:

• [x] Make a tagged release, and list the version tag of the archived version here.
• [x] Archive the reviewed software in Zenodo
• [x] Check the Zenodo deposit has the correct metadata, this includes the title (should match the paper title) and author list (make sure the list is correct and people who only made a small fix are not on it); you may also add the authors' ORCID.
• [x] Please list the Zenodo DOI of the archived version here.

[abhishektha]

@labarba All done:
tagged release: https://github.com/anderson-cu-bioastronautics/dynamo_realsense-capture/releases/tag/v1.1

zenodo DOI:
10.5281/zenodo.3464497

zenodo link: https://zenodo.org/record/3464497

Thanks!

[labarba]

@whedon set v1.1 as version

[whedon]

OK. v1.1 is the version.

[labarba]

@whedon set 10.5281/zenodo.3464497 as archive

[whedon]

OK. 10.5281/zenodo.3464497 is the archive.

[labarba]

@whedon accept

[whedon]

Attempting dry run of processing paper acceptance...

[whedon]

```Reference check summary:

OK DOIs

• 10.1016/j.displa.2013.08.011 is OK
• 10.1016/j.jbiomech.2008.06.024 is OK
• 10.1007/978-0-387-31439-6_472 is OK
• 10.1107/S0567739476001873 is OK

MISSING DOIs

• None

INVALID DOIs

• None
  ```

[whedon]

Check final proof :point_right: https://github.com/openjournals/joss-papers/pull/986

If the paper PDF and Crossref deposit XML look good in https://github.com/openjournals/joss-papers/pull/986, then you can now move forward with accepting the submission by compiling again with the flag deposit=true e.g.
@whedon accept deposit=true

[labarba]

@whedon accept deposit=true

[whedon]

Doing it live! Attempting automated processing of paper acceptance...

[whedon]

🐦🐦🐦 👉 Tweet for this paper 👈 🐦🐦🐦

[whedon]

🚨🚨🚨 THIS IS NOT A DRILL, YOU HAVE JUST ACCEPTED A PAPER INTO JOSS! 🚨🚨🚨

Here's what you must now do:

1. Check final PDF and Crossref metadata that was deposited :point_right: https://github.com/openjournals/joss-papers/pull/987
2. Wait a couple of minutes to verify that the paper DOI resolves https://doi.org/10.21105/joss.01466
3. If everything looks good, then close this review issue.

4. Party like you just published a paper! 🎉🌈🦄💃👻🤘

   Any issues? notify your editorial technical team...

[labarba]

Congratulations, @abhishekbajpayee, your JOSS paper is published! 🚀

Thank you and hats off to our reviewers: @melund, @ixjlyons, @alcantarar — JOSS depends on you! 🙏

[whedon]

:tada::tada::tada: Congratulations on your paper acceptance! :tada::tada::tada:

If you would like to include a link to your paper from your README use the following code snippets:

Markdown:
[![DOI](https://joss.theoj.org/papers/10.21105/joss.01466/status.svg)](https://doi.org/10.21105/joss.01466)

HTML:
<a style="border-width:0" href="https://doi.org/10.21105/joss.01466">
  <img src="https://joss.theoj.org/papers/10.21105/joss.01466/status.svg" alt="DOI badge" >
</a>

reStructuredText:
.. image:: https://joss.theoj.org/papers/10.21105/joss.01466/status.svg
   :target: https://doi.org/10.21105/joss.01466

This is how it will look in your documentation:

We need your help!

Journal of Open Source Software is a community-run journal and relies upon volunteer effort. If you'd like to support us please consider doing either one (or both) of the the following:

• Volunteering to review for us sometime in the future. You can add your name to the reviewer list here: https://joss.theoj.org/reviewer-signup.html
• Making a small donation to support our running costs here: https://numfocus.salsalabs.org/donate-to-joss

[alcantarar]

Congratulations, @abhishekbajpayee, your JOSS paper is published! 🚀

Thank you and hats off to our reviewers: @melund, @ixjlyons, @alcantarar — JOSS depends on you! 🙏

@labarba I think you mean "Congrats @abhishektha!"

[abhishektha]

Thank you so much everyone!

[labarba]

@labarba I think you mean "Congrats @abhishektha!"

Yikes. Auto-complete fail.