Submitting author: @vaishnavtv (Venkata Vaishnav Tadiparthi)
Repository: https://github.com/uqLab/stedy
Version: v0.1.0
Editor: @labarba
Reviewer: @Kevin-Mattheus-Moerman, @ctdegroot, @apsabelhaus
Archive: 10.5281/zenodo.2527084
Status badge code:
HTML: <a href="http://joss.theoj.org/papers/932cce456c4dd1514e6c4e46d2088ad7"><img src="http://joss.theoj.org/papers/932cce456c4dd1514e6c4e46d2088ad7/status.svg"></a>
Markdown: [](http://joss.theoj.org/papers/932cce456c4dd1514e6c4e46d2088ad7)
Reviewers and authors:
Please avoid lengthy details of difficulties in the review thread. Instead, please create a new issue in the target repository and link to those issues (especially acceptance-blockers) in the review thread below. (For completists: if the target issue tracker is also on GitHub, linking the review thread in the issue or vice versa will create corresponding breadcrumb trails in the link target.)
@Kevin-Mattheus-Moerman & @ctdegroot, please carry out your review in this issue by updating the checklist below. If you cannot edit the checklist please:
The reviewer guidelines are available here: https://joss.theoj.org/about#reviewer_guidelines. Any questions/concerns please let @labarba know.
✨ Please try and complete your review in the next two weeks ✨
paper.md
file include a list of authors with their affiliations?paper.md
file include a list of authors with their affiliations?paper.md
file include a list of authors with their affiliations?Hello human, I'm @whedon, a robot that can help you with some common editorial tasks. @Kevin-Mattheus-Moerman, it looks like you're currently assigned as the reviewer for this paper :tada:.
:star: Important :star:
If you haven't already, you should seriously consider unsubscribing from GitHub notifications for this (https://github.com/openjournals/joss-reviews) repository. As a reviewer, you're probably currently watching this repository which means for GitHub's default behaviour you will receive notifications (emails) for all reviews 😿
To fix this do the following two things:
For a list of things I can do to help you, just type:
@whedon commands
Attempting PDF compilation. Reticulating splines etc...
@labarba over at https://github.com/openjournals/joss-reviews/issues/1000# @apsabelhaus agreed to be reviewer, perhaps you could add him here.
@apsabelhaus let me introduce you to JOSS and reviewing for JOSS. JOSS is a free, open access, and open source journal. Reviews take place here, i.e. in Github issues. As you can see the top of this issue has checklists for the reviewers which will guide them through the review process. A set of checklists will be created for you if you join as reviewer. As described in our review guidelines the review focuses on the software and a short paper. Reviewers can comment on this work here in this issue and are encouraged to created dedicated issues on the software repository, and link to them here, for larger items. Let me know if you have more questions.
@labarba so far @vaishnavtv has worked on these issues which I've posted on the software repository:
I manually added the header comment to add @apsabelhaus as reviewer — thank you, all.
@Kevin-Mattheus-Moerman : I forgot how to give @apsabelhaus the permissions to edit the reviewer checklist or send him the appropriate invitation. Do you remember? I went to the Teams section of the open journals
org, and couldn't immediately identify a team to add him. Help…
@labarba The first step is to do this:
@whedon add @apsabelhaus as reviewer
Then the reviewer should accept this invitation: https://github.com/openjournals/joss-reviews/invitations
The @whedon add
commands sets things up _only_ if used in the Pre-Review issue, _before_ starting the review. Once we're here, adding a reviewer is a manual process.
You can find the pre-print of the cited reference here.
I hope the review is going smoothly. Please let us know if anything else is required.
Any issues with the review? It has been almost two weeks since the last update. Kindly let us know if there are problems with the software.
@vaishnavtv Please have patience. As you know peer review takes time and reviewers cannot drop everything immediately to tend to a review. I will review within the next two weeks.
@vaishnavtv I kindly ask you that you be patient, and allow the review to proceed at a pace that reflects the fact that everyone here is a volunteer.
@vaishnavtv In the paper, the affiliations are incomplete. The position of each author (i.e. Graduate Research Assistant or Associate Professor) is not part of the affiliation and is not required. Additionally, provide more details on affiliation (i.e. department/faculty, city, state, country).
@vaishnavtv The paper contains insufficient references and the one reference that is included is not formatted properly (it appears twice). Please include at least a couple more references. For example, you mention some similar software called "Simscape Multibody". You should reference that. The first paragraph needs a reference or two also.
@vaishnavtv Where can I find your automated tests?
@whedon generate pdf
Attempting PDF compilation. Reticulating splines etc...
PDF failed to compile for issue #1042 with the following error:
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0
100 15 0 15 0 0 252 0 --:--:-- --:--:-- --:--:-- 254
Error reading bibliography ./JOSSpaper.bib (line 6, column 3):
unexpected "y"
expecting space, ",", white space or "}"
Error running filter pandoc-citeproc:
Filter returned error status 1
Looks like we failed to compile the PDF
@whedon generate pdf
Attempting PDF compilation. Reticulating splines etc...
@ctdegroot I have made the recommended changes to the paper - affiliations and references have been added. We were under the impression that our manuscript submitted for publication would suffice as it exhaustively covers all information pertinent to this submission as well. Thank you for the clarification.
For the automated tests, would the files provided in the Examples folder be sufficient?
@vaishnavtv The paper looks good now.
I don't think that the files in the Examples folder are sufficient as test cases because they do not assert what result is expected. Let's imagine that I am a user that makes some changes/extensions to the code. How to I check that I haven't broken anything that already existed?
From the JOSS review criteria:
Good: An automated test suite hooked up to an external service such as Travis-CI or similar
OK: Documented manual steps that can be followed to check the expected functionality of the software (e.g. a sample input file to assert behaviour)
Bad (not acceptable): No way for you the reviewer to check whether the software works
To get the tests into the 'OK' category you will need to provide some reference results that can be checked (preferably automatically) with the code outputs.
One other note, is that you should remove all of the ".DS_Store" files that are in some of the example folders. Normally you can put this in your .gitignore file to prevent them from being tracked in the future.
@ctdegroot
As recommended, I have added a Tests folder to the repository. The user can simply run TestPendulum.m to check the results of their changes to the code. I have also modified the Contributing Guidelines document to reflect this update.
I have removed all the .DS_Store files as well.
@Kevin-Mattheus-Moerman — Your last post here was 26 days ago. At this time, can I ask you for an ETA on your review?
@ctdegroot — Are you satisfied with the revisions? You have several items to check off your review list, still.
@apsabelhaus — We haven't heard from you. Will you still be able to contribute a review of this submission? Let me know if there is any way I can help you.
@labarba I'm satisfied with the changes so far. Still need to finish checking on the functionality of the code; hopefully soon.
@vaishnavtv can you please work on adding basic community guidelines? See this issue:
@labarba thanks for the reminder. Only one box now remains unticked and it relates to community guidelines. I've opened an issue on this an am awaiting feedback from the author.
@labarba I've just ticked the last box and recommend that this work is accepted. Thanks for the opportunity to review this interesting work.
We have recommendations to accept from @Kevin-Mattheus-Moerman and @ctdegroot, but we never heard back from @apsabelhaus.
👋 @apsabelhaus — did this escape your radar? If you're not available now to volunteer a review, we can proceed without you, and perhaps count on you at some other time. You can sign up online to be listed as a potential reviewer.
@labarba yes, this did indeed fall under my radar! My apologies to the review team and the authors. I'll perform the review today.
@Kevin-Mattheus-Moerman it doesn't seem like I'm able to fill out the checklist. Following the link to https://github.com/openjournals/joss-reviews/invitations seems to imply I haven't been formally sent an invitation. Could you do so / am I missing something? I'll proceed with evaluating the software in the meantime! Thanks again for everyone's patience, I'm more used to traditional academic peer-review deadlines.
@arfon Can you help us? I added @apsabelhaus as reviewer manually here, after the review started, so we don't have the benefit of our bot whedon
setting the permissions right. I don't remember how to give the permissions to the reviewer, in this case. I looked at the teams in the org, but it's not clear to me: a joss-reviewers
team exists, but it has only two members.
@labarba - I've added @apsabelhaus as an external collaborator to this repository which will will allow them to update the checklist. @apsabelhaus - the invite is available here: https://github.com/openjournals/joss-reviews/invitations
Thank you @arfon and @labarba ! Started checking off the list.
My initial review is that this is a very nice piece of software, with a well-defined use case, filling a gap that the community needs. With some minor changes and clarifications, I would be ready to recommend for publication. @vaishnavtv , I've opened a handful of issues, mostly about documentation or technical questions. I've currently left the following boxes unticked for these reasons:
1) Functionality: one minor potential bug about the video.
2) Functionality documentation: some confusion about the API and technical assumptions.
4) References: just a bit more explanation needed about certain technical parts that weren't referred to in the JOSS paper or your preprint. I've provided some examples to refer to.
Thanks to the authors for a great contribution to the tensegrity systems community!
@vaishnavtv You need to revise your installation instructions. It is not clear that steps 3b and 3c are only required if there is an error. I had to read several times to make sense of it.
@vaishnavtv In the test that you've added, please do not make the user interactively enter data. If I'm trying to test the code as a new user, I don't know what I should enter. Is one test enough to verify the full functioning of the code?
@vaishnavtv Regarding the item "Performance: If there are any performance claims of the software, have they been confirmed? (If there are no claims, please check off this item.)": You have stated in your paper that "We have demonstrated superiority of the method in terms of accuracy over the commercially avail- able dynamics simulator, Simscape Multibody". There is a performance claim made. How do I confirm it?
@labarba I'm done with my review for now. I'm not yet satisfied with the automated tests. There is only one test and does not seem to cover the code well enough to verify its functionality. It also does not seem to have a reference solution to compare to, it is just checking some energy constraints. I assume by automated tests, we are talking about unit tests, not just integration tests? If the author writes an actual set of unit tests that covers the code to a reasonable degree (i.e. at the function level) I'd be satisfied.
Thank you all for your careful work!
@ctdegroot We don't _require_ test coverage or unit tests vs. regression tests.
The JOSS Review criteria on tests are:
Authors are strongly encouraged to include an automated test suite covering the core functionality of their software.
Good: An automated test suite hooked up to an external service such as Travis-CI or similar
OK: Documented manual steps that can be followed to check the expected functionality of the software (e.g. a sample input file to assert behaviour)
Bad (not acceptable): No way for you the reviewer to check whether the software works
@labarba Thank you for the clarification. In this case I still think it falls into the not acceptable category. I ran the one test that was provided and based on the output I don't know if the code is working or not. It runs and shows some plots, but is it working correctly? Not sure.
@ctdegroot, when you run the test case, you should see the results of 3 checks (bar length, energy constraints and motion errors) in the command window. Let me know if that’s not being displayed.
@ctdegroot , about the performance claim issue, we demonstrated superiority in terms of accuracy over Simscape multibody and attached plots in the preprint. Would you like us to attach the relevant files to the repository?
@vaishnavtv The preprint is reference in the paper right? That is good enough for me to verify the performance claim. For the test, what are you comparing to for a reference? Is it an exact solution? It is not clear to me from inspecting the code or running the case. There are no comments in "Pendulum_MinRealODE.m" so I don't know what it does.
@ctdegroot The test compares the results of our simulation with that of the commonly accepted ground truth for the same structure, i.e., a pendulum simulation performed using minimum coordinates. I will add comments to the corresponding files with a proper description. Would that be sufficient?
@vaishnavtv If you are content with doing the absolute minimum possible amount of testing then I suppose, since the other reviewers have accepted your tests, that I can check the item off.
@ctdegroot, as recommended I have added another test case to the repository. I'm amending the contributing guidelines to reflect this change.
@vaishnavtv Thanks for fixing these up. I've ticked the box for tests now, since there is enough to check the code to a reasonable degree.
@ctdegroot, thank you for the helpful suggestions all throughout the review.
@apsabelhaus, I have made all the recommended changes to the documentation. Kindly let us know if you wanted us to include anything else.
My review is concluded. I'm satisfied with the changes.
@vaishnavtv : I read over your revised paper and I only have one suggestion, and that is to delete the sentence: “The scientific computing community’s familiarity with MATLAB makes it a favored choice for performing simulations of this nature.” (Unsubstantiated assessment.)
We now only await the recommendation to accept from @apsabelhaus.
Unofficially, I was surprised that you chose to deposit your preprint on ResearchGate, rather than arXiv, or engrXiv. I encourage you to consider these options in the future.
@whedon generate pdf
Attempting PDF compilation. Reticulating splines etc...
@labarba statement deleted.
Could you help explain to me why one should choose arXiv or any of the other OSF journals over ResearchGate to publish preprints? Is there a visibility advantage?
Thanks for asking! In mathematics, physics and computer science, arXiv is _a way of life_. Many researchers read the arXiv mailing lists of new deposits daily. In engineering, it's less so, but a budding dedicated engineering alternative is engrXiv—this is a non-profit, researcher-led effort (I'm in the steering committee). OSF is also a non-profit, with funding from charitable foundations, and led by researchers.
RG is a for-profit company, funded by venture capital, and many leaders of the open science movement have been critical of their ... ahem ... practices. For years, RG illegally hosted articles, to make profit (not sure what the situation is now). They infamously spam and click-bait researchers to their site. Check more of the criticisms on Wikipedia.
I don't maintain my RG profile but let it automatically show the publications it finds online for me.
@apsabelhaus : You have a few unchecked items, still. Have the authors addressed your comments?
👋 @apsabelhaus — We're waiting for you to finish the checklist. Are you still working on your review? Or are you ready to make a recommendation?
Hi all, thanks again for your patience, the semester just finished up here, so I'm going to work on this now.
Hi all - an update from me. Almost ready for publication, only item left is 'functionality.' There's one last, potentially significant, issue to be addressed. I'll keep a close eye on this review throughout the week so the authors can get it done before the holidays.
All boxes checked from me. I recommend publication.
@whedon accept
No archive DOI set. Exiting...
Oops. Forgot one thing ... @vaishnavtv: Could you now make a deposit in an archive, such as Zenodo, and give us the DOI here?
@whedon set 10.5281/zenodo.2527084 as archive
OK. 10.5281/zenodo.2527084 is the archive.
👋 @vaishnavtv — Could you edit the metadata over at Zenodo, so that the title and list of authors match this submission?
Hi there, @vaishnavtv — As soon as you correct the metadata of the Zenodo entry, I can get you paper published here in JOSS. Let me know!
Happy new year!
@labarba, it’s done.
Is there anything else we need to add?
@whedon accept
Attempting dry run of processing paper acceptance...
Check final proof :point_right: https://github.com/openjournals/joss-papers/pull/419
If the paper PDF and Crossref deposit XML look good in https://github.com/openjournals/joss-papers/pull/419, then you can now move forward with accepting the submission by compiling again with the flag deposit=true
e.g.
@whedon accept deposit=true
@whedon accept deposit=true
Doing it live! Attempting automated processing of paper acceptance...
🚨🚨🚨 THIS IS NOT A DRILL, YOU HAVE JUST ACCEPTED A PAPER INTO JOSS! 🚨🚨🚨
Here's what you must now do:
Party like you just published a paper! 🎉🌈🦄💃👻🤘
Any issues? notify your editorial technical team...
Congratulations, @vaishnavtv — your paper is published, and is the first JOSS paper of 2019!
And a big, happy Thank You to our reviewers: @Kevin-Mattheus-Moerman, @ctdegroot, @apsabelhaus
:tada::tada::tada: Congratulations on your paper acceptance! :tada::tada::tada:
If you would like to include a link to your paper from your README use the following code snippets:
Markdown:
[](https://doi.org/10.21105/joss.01042)
HTML:
<a style="border-width:0" href="https://doi.org/10.21105/joss.01042">
<img src="http://joss.theoj.org/papers/10.21105/joss.01042/status.svg" alt="DOI badge" >
</a>
reStructuredText:
.. image:: http://joss.theoj.org/papers/10.21105/joss.01042/status.svg
:target: https://doi.org/10.21105/joss.01042
This is how it will look in your documentation:
We need your help!
Journal of Open Source Software is a community-run journal and relies upon volunteer effort. If you'd like to support us please consider doing either one (or both) of the the following:
Thank you @labarba and to all the reviewers, for your helpful inputs in the review process!
Most helpful comment
Congratulations, @vaishnavtv — your paper is published, and is the first JOSS paper of 2019!
And a big, happy Thank You to our reviewers: @Kevin-Mattheus-Moerman, @ctdegroot, @apsabelhaus