Joss-reviews: [REVIEW]: DART: Dynamic Animation and Robotics Toolkit

Created on 12 Dec 2017  ·  52Comments  ·  Source: openjournals/joss-reviews

Submitting author: @jslee02 (Jeongseok Lee)
Repository: https://github.com/dartsim/dart
Version: 6.3.0
Editor: @Kevin-Mattheus-Moerman
Reviewer: @costashatz
Archive: 10.5281/zenodo.1166142

Status

status

Status badge code:

HTML: <a href="https://joss.theoj.org/papers/f0cc65c89f6bbb55a35be08189e32bd2"><img src="https://joss.theoj.org/papers/f0cc65c89f6bbb55a35be08189e32bd2/status.svg"></a>
Markdown: [![status](https://joss.theoj.org/papers/f0cc65c89f6bbb55a35be08189e32bd2/status.svg)](https://joss.theoj.org/papers/f0cc65c89f6bbb55a35be08189e32bd2)

Reviewers and authors:

Please avoid lengthy details of difficulties in the review thread. Instead, please create a new issue in the target repository and link to those issues (especially acceptance-blockers) in the review thread below. (For completists: if the target issue tracker is also on GitHub, linking the review thread in the issue or vice versa will create corresponding breadcrumb trails in the link target.)

## Reviewer instructions & questions

REVIEWER 1

@bmagyar, please carry out your review in this issue by updating the checklist below. If you cannot edit the checklist please:

  1. Make sure you're logged in to your GitHub account
  2. Be sure to accept the invite at this URL: https://github.com/openjournals/joss-reviews/invitations

The reviewer guidelines are available here: https://joss.theoj.org/about#reviewer_guidelines. Any questions/concerns please let @Kevin-Mattheus-Moerman know.

Conflict of interest

Code of Conduct

General checks

  • [x] Repository: Is the source code for this software available at the repository url?
  • [x] License: Does the repository contain a plain-text LICENSE file with the contents of an OSI approved software license?
  • [x] Version: Does the release version given match the GitHub release (6.3.0)?
  • [x] Authorship: Has the submitting author (@jslee02) made major contributions to the software? Does the full list of paper authors seem appropriate and complete?

Functionality

  • [x] Installation: Does installation proceed as outlined in the documentation?
  • [x] Functionality: Have the functional claims of the software been confirmed?
  • [x] Performance: If there are any performance claims of the software, have they been confirmed? (If there are no claims, please check off this item.)

Documentation

  • [x] A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
  • [x] Installation instructions: Is there a clearly-stated list of dependencies? Ideally these should be handled with an automated package management solution.
  • [x] Example usage: Do the authors include examples of how to use the software (ideally to solve real-world analysis problems).
  • [x] Functionality documentation: Is the core functionality of the software documented to a satisfactory level (e.g., API method documentation)?
  • [x] Automated tests: Are there automated tests or manual steps described so that the function of the software can be verified?
  • [x] Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support

Software paper

  • [x] Authors: Does the paper.md file include a list of authors with their affiliations?
  • [x] A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
  • [x] References: Do all archival references that should have a DOI list one (e.g., papers, datasets, software)?

REVIEWER 2

@betatim, please carry out your review in this issue by updating the checklist below. If you cannot edit the checklist please:

  1. Make sure you're logged in to your GitHub account
  2. Be sure to accept the invite at this URL: https://github.com/openjournals/joss-reviews/invitations

The reviewer guidelines are available here: https://joss.theoj.org/about#reviewer_guidelines. Any questions/concerns please let @Kevin-Mattheus-Moerman know.

Conflict of interest

Code of Conduct

General checks

  • [x] Repository: Is the source code for this software available at the repository url?
  • [x] License: Does the repository contain a plain-text LICENSE file with the contents of an OSI approved software license?
  • [x] Version: Does the release version given match the GitHub release (6.3.0)?
  • [ ] Authorship: Has the submitting author (@jslee02) made major contributions to the software? Does the full list of paper authors seem appropriate and complete?

Functionality

  • [ ] Installation: Does installation proceed as outlined in the documentation?
  • [ ] Functionality: Have the functional claims of the software been confirmed?
  • [ ] Performance: If there are any performance claims of the software, have they been confirmed? (If there are no claims, please check off this item.)

Documentation

  • [ ] A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
  • [ ] Installation instructions: Is there a clearly-stated list of dependencies? Ideally these should be handled with an automated package management solution.
  • [ ] Example usage: Do the authors include examples of how to use the software (ideally to solve real-world analysis problems).
  • [ ] Functionality documentation: Is the core functionality of the software documented to a satisfactory level (e.g., API method documentation)?
  • [ ] Automated tests: Are there automated tests or manual steps described so that the function of the software can be verified?
  • [ ] Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support

Software paper

  • [ ] Authors: Does the paper.md file include a list of authors with their affiliations?
  • [ ] A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
  • [ ] References: Do all archival references that should have a DOI list one (e.g., papers, datasets, software)?

REVIEWER 3

@costashatz, please carry out your review in this issue by updating the checklist below. If you cannot edit the checklist please:

  1. Make sure you're logged in to your GitHub account
  2. Be sure to accept the invite at this URL: https://github.com/openjournals/joss-reviews/invitations

The reviewer guidelines are available here: https://joss.theoj.org/about#reviewer_guidelines. Any questions/concerns please let @Kevin-Mattheus-Moerman know.

Conflict of interest

Code of Conduct

General checks

  • [x] Repository: Is the source code for this software available at the repository url?
  • [x] License: Does the repository contain a plain-text LICENSE file with the contents of an OSI approved software license?
  • [x] Version: Does the release version given match the GitHub release (6.3.0)?
  • [x] Authorship: Has the submitting author (@jslee02) made major contributions to the software? Does the full list of paper authors seem appropriate and complete?

Functionality

  • [x] Installation: Does installation proceed as outlined in the documentation?
  • [x] Functionality: Have the functional claims of the software been confirmed?
  • [x] Performance: If there are any performance claims of the software, have they been confirmed? (If there are no claims, please check off this item.)

Documentation

  • [x] A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
  • [x] Installation instructions: Is there a clearly-stated list of dependencies? Ideally these should be handled with an automated package management solution.
  • [x] Example usage: Do the authors include examples of how to use the software (ideally to solve real-world analysis problems).
  • [x] Functionality documentation: Is the core functionality of the software documented to a satisfactory level (e.g., API method documentation)?
  • [x] Automated tests: Are there automated tests or manual steps described so that the function of the software can be verified?
  • [x] Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support

Software paper

  • [x] Authors: Does the paper.md file include a list of authors with their affiliations?
  • [x] A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
  • [x] References: Do all archival references that should have a DOI list one (e.g., papers, datasets, software)?
accepted published recommend-accept review
Source
picture whedon

All 52 comments

Hello human, I'm @whedon. I'm here to help you with some common editorial tasks. @bmagyar it looks like you're currently assigned as the reviewer for this paper :tada:.

:star: Important :star:

If you haven't already, you should seriously consider unsubscribing from GitHub notifications for this (https://github.com/openjournals/joss-reviews) repository. As a reviewer, you're probably currently watching this repository which means for GitHub's default behaviour you will receive notifications (emails) for all reviews 😿

To fix this do the following two things:

  1. Set yourself as 'Not watching' https://github.com/openjournals/joss-reviews:

watching

  1. You may also like to change your default settings for this watching repositories in your GitHub profile here: https://github.com/settings/notifications

notifications

For a list of things I can do to help you, just type:

@whedon commands
whedon picture whedon on 12 Dec 2017 
Attempting PDF compilation. Reticulating splines etc...
whedon picture whedon on 12 Dec 2017

@jslee02 @bmagyar @betatim let the reviewing begin!

@betatim you will have to accept the invitation to join the JOSS reviewers community: https://github.com/openjournals/joss-reviews/invitations before you will be able to check off items. Once you do so, I will also assign you to the GitHub issue. @arfon can you process the invitation?

Please use the two checklists above (one for each of you) to carry out your reviews.

An informal guideline is that we would like your review in 2 weeks, but sooner (or later) are also ok.

The reviewer guidelines are available here: http://joss.theoj.org/about#reviewer_guidelines

Please avoid lengthy details of difficulties in this review thread. Instead, please create a new issue in the target repository and link to those issues (especially acceptance-blockers) in this review thread. (For completists: if the target issue tracker is also on GitHub, linking the review thread in the issue or vice versa will create corresponding breadcrumb trails in the link target.)

Any questions/concerns, please let me know.

Thanks!!

Kevin-Mattheus-Moerman picture Kevin-Mattheus-Moerman on 12 Dec 2017

@betatim - you should have permissions to update this checklist once you accept the invite at https://github.com/openjournals/joss-reviews/invitations

arfon picture arfon on 12 Dec 2017

@whedon generate pdf

arfon picture arfon on 12 Dec 2017 
Attempting PDF compilation. Reticulating splines etc...
whedon picture whedon on 12 Dec 2017 
https://github.com/openjournals/joss-papers/blob/joss.00500/joss.00500/10.21105.joss.00500.pdf
whedon picture whedon on 12 Dec 2017 
https://github.com/openjournals/joss-papers/blob/joss.00500/joss.00500/10.21105.joss.00500.pdf
whedon picture whedon on 12 Dec 2017

Almost all green from me, excellent submission :+1:

bmagyar picture bmagyar on 3 Jan 2018

@whedon generate pdf

jslee02 picture jslee02 on 5 Jan 2018 
Attempting PDF compilation. Reticulating splines etc...
whedon picture whedon on 5 Jan 2018 
https://github.com/openjournals/joss-papers/blob/joss.00500/joss.00500/10.21105.joss.00500.pdf
whedon picture whedon on 5 Jan 2018

@bmagyar thanks for reviewing! All your boxes are ticked it seems. Is it safe to assume you are happy with this submission and recommend that we accept it?

@betatim how are you getting on?

Kevin-Mattheus-Moerman picture Kevin-Mattheus-Moerman on 8 Jan 2018

@jslee02 I have some minor comments on your paper.

  1. Could you rephrase the heading "Research Publications Utilized DART"? Perhaps "Research publications utilizing DART" or "Research publications which utilized DART" sounds better.

  2. Check the sentence: "In addition, DART gives provides flexibility...", Looks like removal of the word "gives" would fix this.

Kevin-Mattheus-Moerman picture Kevin-Mattheus-Moerman on 8 Jan 2018

@Kevin-Mattheus-Moerman Resolved. Thanks for the corrections.

jslee02 picture jslee02 on 8 Jan 2018

@whedon generate pdf

jslee02 picture jslee02 on 8 Jan 2018 
Attempting PDF compilation. Reticulating splines etc...
whedon picture whedon on 8 Jan 2018 
https://github.com/openjournals/joss-papers/blob/joss.00500/joss.00500/10.21105.joss.00500.pdf
whedon picture whedon on 8 Jan 2018

@Kevin-Mattheus-Moerman yes, green light all the way from me! :+1:

bmagyar picture bmagyar on 9 Jan 2018

I've not put in the time to get the software working/installed on my Mac. It is unfortunately also not anywhere near the top of my todo list. So if we can find someone else who wants to take over I would be A++ super Ok with that.

betatim picture betatim on 19 Jan 2018

Three more papers were added to the "Research publications utilizing DART" section by https://github.com/dartsim/dart/issues/957.

jslee02 picture jslee02 on 22 Jan 2018

@whedon generate pdf

jslee02 picture jslee02 on 22 Jan 2018 
Attempting PDF compilation. Reticulating splines etc...
whedon picture whedon on 22 Jan 2018 
https://github.com/openjournals/joss-papers/blob/joss.00500/joss.00500/10.21105.joss.00500.pdf
whedon picture whedon on 22 Jan 2018

@Kevin-Mattheus-Moerman It seems @betatim is not available to review this submission at this moment (https://github.com/openjournals/joss-reviews/issues/500#issuecomment-358904542). Please let me know if there is anything I can do for proceeding.

jslee02 picture jslee02 on 25 Jan 2018

@Kevin-Mattheus-Moerman I could volunteer to review the library (if a second reviewer is needed). I've been using it quite a lot, but I have no connection with the developers (I am not sure about the conflict of interest policy). Cheers!

costashatz picture costashatz on 25 Jan 2018

@costashatz yes that would be great! I'll add a form for you to tick boxes.

Kevin-Mattheus-Moerman picture Kevin-Mattheus-Moerman on 25 Jan 2018

@whedon set @costashatz as reviewer

Kevin-Mattheus-Moerman picture Kevin-Mattheus-Moerman on 25 Jan 2018

@whedon commands

Kevin-Mattheus-Moerman picture Kevin-Mattheus-Moerman on 25 Jan 2018

Here are some things you can ask me to do:

# List all of Whedon's capabilities
@whedon commands

# Assign a GitHub user as the reviewer of this submission
@whedon assign @username as reviewer

# List of editor GitHub usernames
@whedon list editors

# List of reviewers together with programming language preferences and domain expertise
@whedon list reviewers

# Change editorial assignment
@whedon assign @username as editor

# Set the software archive DOI at the top of the issue e.g.
@whedon set 10.0000/zenodo.00000 as archive

# Open the review issue
@whedon start review

🚧 🚧 🚧 Experimental Whedon features 🚧 🚧 🚧

# Compile the paper
@whedon generate pdf
whedon picture whedon on 25 Jan 2018

@whedon assign @costashatz as reviewer

Kevin-Mattheus-Moerman picture Kevin-Mattheus-Moerman on 25 Jan 2018

OK, the reviewer is @costashatz

whedon picture whedon on 25 Jan 2018

@costashatz thanks for helping out here. I've added a REVIEWER 3 section at the top here. This is where you will work to tick boxes. You should be able to tick them (Be sure to accept the invite at this URL: https://github.com/openjournals/joss-reviews/invitations). Let me know if you have questions.

Kevin-Mattheus-Moerman picture Kevin-Mattheus-Moerman on 25 Jan 2018

@betatim I've left your review tickbox set in tact in case you do find time to contribute to the review. Either way thanks for your input.

Kevin-Mattheus-Moerman picture Kevin-Mattheus-Moerman on 25 Jan 2018

@costashatz thanks again for jumping in to save the day 🚀 ! Could you give me an indication as to when you expect to finalize the review? Thanks

Kevin-Mattheus-Moerman picture Kevin-Mattheus-Moerman on 25 Jan 2018

@costashatz thanks again for jumping in to save the day ! Could you give me an indication as to when you expect to finalize the review? Thanks

No problem! Since I am already using the library (i.e., I have it installed, already done the tutorials etc), I think it'll be quick!

costashatz picture costashatz on 25 Jan 2018

@jslee02 Great library and submission! Only two small remarks:

  • Is there a specific reason why you do not have API docs for v6.3.0 that is being submitted? I am aware the API didn't change much from v6.1.1 to v6.3.0, but I'd guess it's not difficult to generate it right?
  • Regarding the performance, as I already noted in this issue there seems to be something slowing down the simulation with collisions (mainly the one of FCL) even with simple cases (like the collision/dominoes tutorial). Have you identified what the problem is? Does it come from DART or FCL?

Both of my remarks are minor as the library is very well written and documented!

Thanks!

costashatz picture costashatz on 25 Jan 2018

@costashatz Thank you for the quick feedback. Please find below my answers to your remarks:

  • Updated API docs can be found on the website or API repo.

  • Regarding the collision/contact performance, I replied on the issue:

This is somewhat an expected result. There is two main reasons behind this: (1) collision handling method in DART and (2) number of contact points generated by the collision detector.

(1) DART uses rather slow but more precise contact constraint solver.

AFAIK, there are two major categories of solving non-penetration contact constraint handling methods: (a) LCP-based method and (b) iterative method. I believe DART initially chose (a) method to compute precise contact points sacrificing speed, but we are working on supporting (b) for interactive simulations like manipulation in VR. This is my working branch hoping to finish initial implementation by next couple of months.

(2) FCLCollisionDetector generally returns more contact points than DARTCollisionDetector.

In FCL setting of DART, we use meshes even for primitive shapes (e.g., box and sphere) by default, and this generally results in more contact points than primitive-primitive shape pair contacts (like DARTCollisionDetector). This is because FCL detects all the contact points of the meshes' triangle pairs in contact while primitive-primitive collision algorithm returns minimally required contacts (this can be varied by the concrete primitive collision algorithm). For example, box-box collision algorithm returns up to four contacts.

Loosely speaking, the complexity of (a) method is O(n^3) where n is the number of contacts (assuming no friction). That said, more contacts make the simulate slow quickly.

The reason we don't use FCL's primitive shape support is that it has a few unresolved problems. I had worked on addressing those issues (e.g., https://github.com/flexible-collision-library/fcl/pull/52, https://github.com/flexible-collision-library/fcl/pull/61), but there are some remains.

There is persistent contact manifold technic of Bullet Physics engine that can address this issue. It manages the contact points reported from the collision detector to be less than 4, which is minimal contact points to support the contact shapes each other. I'm also considering having the similar feature in DART.

jslee02 picture jslee02 on 26 Jan 2018

@costashatz Thank you for the quick feedback. Please find below my answers to your remarks:

Updated API docs can be found on the website or API repo.

Regarding the collision/contact performance, I replied on the issue

@jslee02 Thank you for your fast answers (I will continue the discussion in the issue regarding the collision/contact performance). I am very much satisfied by the library and the submission! Best of luck!

@Kevin-Mattheus-Moerman Let me know if you need me to do anything else..

costashatz picture costashatz on 31 Jan 2018
🎉1

@Kevin-Mattheus-Moerman Sorry if I've made you feel like I'm putting too much pressure on you, but do you think there is anything left needs to be done? It would be great if I could provide the way to cite DART soon to the authors used DART for their research.

jslee02 picture jslee02 on 5 Feb 2018

@whedon generate pdf

Kevin-Mattheus-Moerman picture Kevin-Mattheus-Moerman on 5 Feb 2018 
Attempting PDF compilation. Reticulating splines etc...
whedon picture whedon on 5 Feb 2018 
https://github.com/openjournals/joss-papers/blob/joss.00500/joss.00500/10.21105.joss.00500.pdf
whedon picture whedon on 5 Feb 2018

@jslee02 my apologies for the delay in getting back to you! I recommend that this paper is accepted. At this point please provide a DOI for the repository of the final reviewed version. @arfon will then proceed with formal acceptance of this paper. 🚀 🤖 🎉

Kevin-Mattheus-Moerman picture Kevin-Mattheus-Moerman on 5 Feb 2018
🎉1

@betatim @costashatz @bmagyar thanks for reviewing this submission! 👏

Kevin-Mattheus-Moerman picture Kevin-Mattheus-Moerman on 5 Feb 2018
👍1

@Kevin-Mattheus-Moerman Thank you! I've created a DOI for DART 6.3.0 here:
https://zenodo.org/record/1166142#.WnjLInXwbhw

jslee02 picture jslee02 on 5 Feb 2018

Great! @arfon we are good to go here 🤖

Kevin-Mattheus-Moerman picture Kevin-Mattheus-Moerman on 5 Feb 2018

@whedon set 10.5281/zenodo.1166142 as archive

arfon picture arfon on 6 Feb 2018

OK. 10.5281/zenodo.1166142 is the archive.

whedon picture whedon on 6 Feb 2018

@jslee02 @bmagyar @betatim - many thanks for your reviews here and to @Kevin-Mattheus-Moerman for editing this submission ✨

@jslee02 - your paper is now accepted into JOSS and your DOI is https://doi.org/10.21105/joss.00500 ⚡️ 🚀 💥

arfon picture arfon on 6 Feb 2018

:tada::tada::tada: Congratulations on your paper acceptance! :tada::tada::tada:

If you would like to include a link to your paper from your README use the following code snippet:

[![DOI](http://joss.theoj.org/papers/10.21105/joss.00500/status.svg)](https://doi.org/10.21105/joss.00500)

This is how it will look in your documentation:

DOI

We need your help!

Journal of Open Source Software is a community-run journal and relies upon volunteer effort. If you'd like to support us please consider either:

  1. Volunteering to review for us sometime in the future. You can add your name to the reviewer list here: http://joss.theoj.org/reviewer-signup.html
  2. Making a small donation to support our running costs here: https://www.flipcause.com/secure/cause_pdetails/Mjk3Nzk=
whedon picture whedon on 6 Feb 2018

Thank you @bmagyar, @betatim, @costashatz for the helpful review; and @Kevin-Mattheus-Moerman, @arfon for coordinating this process! :tada:

jslee02 picture jslee02 on 6 Feb 2018
Was this page helpful?
0 / 5 - 0 ratings

Related issues

whedon picture whedon  ·  6Comments
whedon picture whedon  ·  12Comments
whedon picture whedon  ·  12Comments
whedon picture whedon  ·  8Comments
whedon picture whedon  ·  13Comments