Joss-reviews: [REVIEW]: ScenTrees.jl: A Julia Library For Generating Scenario Trees and Scenario Lattices for Multistage Stochastic Optimization

[whedon]

Submitting author: @kirui93 (Kipngeno )
Repository: https://github.com/kirui93/ScenTrees.jl
Version: v0.2.2
Editor: @melissawm
Reviewer: @juliohm, @matbesancon
Archive: 10.5281/zenodo.3672205

Status

Status badge code:

HTML: <a href="https://joss.theoj.org/papers/5a8cf39444c8e9f6a587ac4361de52c1"><img src="https://joss.theoj.org/papers/5a8cf39444c8e9f6a587ac4361de52c1/status.svg"></a>
Markdown: [![status](https://joss.theoj.org/papers/5a8cf39444c8e9f6a587ac4361de52c1/status.svg)](https://joss.theoj.org/papers/5a8cf39444c8e9f6a587ac4361de52c1)

Reviewers and authors:

Please avoid lengthy details of difficulties in the review thread. Instead, please create a new issue in the target repository and link to those issues (especially acceptance-blockers) by leaving comments in the review thread below. (For completists: if the target issue tracker is also on GitHub, linking the review thread in the issue or vice versa will create corresponding breadcrumb trails in the link target.)

Reviewer instructions & questions

@juliohm & @matbesancon, please carry out your review in this issue by updating the checklist below. If you cannot edit the checklist please:

1. Make sure you're logged in to your GitHub account
2. Be sure to accept the invite at this URL: https://github.com/openjournals/joss-reviews/invitations

The reviewer guidelines are available here: https://joss.readthedocs.io/en/latest/reviewer_guidelines.html. Any questions/concerns please let @melissawm know.

✨ Please try and complete your review in the next two weeks ✨

Review checklist for @juliohm

Conflict of interest

• [x] I confirm that I have read the JOSS conflict of interest (COI) policy and that: I have no COIs with reviewing this work or that any perceived COIs have been waived by JOSS for the purpose of this review.

Code of Conduct

• [x] I confirm that I read and will adhere to the JOSS code of conduct.

General checks

• [x] Repository: Is the source code for this software available at the repository url?
• [x] License: Does the repository contain a plain-text LICENSE file with the contents of an OSI approved software license?
• [x] Contribution and authorship: Has the submitting author (@kirui93) made major contributions to the software? Does the full list of paper authors seem appropriate and complete?

Functionality

• [x] Installation: Does installation proceed as outlined in the documentation?
• [x] Functionality: Have the functional claims of the software been confirmed?
• [x] Performance: If there are any performance claims of the software, have they been confirmed? (If there are no claims, please check off this item.)

Documentation

• [x] A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
• [x] Installation instructions: Is there a clearly-stated list of dependencies? Ideally these should be handled with an automated package management solution.
• [x] Example usage: Do the authors include examples of how to use the software (ideally to solve real-world analysis problems).
• [x] Functionality documentation: Is the core functionality of the software documented to a satisfactory level (e.g., API method documentation)?
• [x] Automated tests: Are there automated tests or manual steps described so that the functionality of the software can be verified?
• [x] Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support

Software paper

• [x] Summary: Has a clear description of the high-level functionality and purpose of the software for a diverse, non-specialist audience been provided?
• [x] A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
• [x] State of the field: Do the authors describe how this software compares to other commonly-used packages?
• [x] Quality of writing: Is the paper well written (i.e., it does not require editing for structure, language, or writing quality)?
• [x] References: Is the list of references complete, and is everything cited appropriately that should be cited (e.g., papers, datasets, software)? Do references in the text use the proper citation syntax?

Review checklist for @matbesancon

Conflict of interest

• [x] I confirm that I have read the JOSS conflict of interest (COI) policy and that: I have no COIs with reviewing this work or that any perceived COIs have been waived by JOSS for the purpose of this review.

Code of Conduct

• [x] I confirm that I read and will adhere to the JOSS code of conduct.

General checks

• [x] Repository: Is the source code for this software available at the repository url?
• [x] License: Does the repository contain a plain-text LICENSE file with the contents of an OSI approved software license?
• [x] Contribution and authorship: Has the submitting author (@kirui93) made major contributions to the software? Does the full list of paper authors seem appropriate and complete?

Functionality

• [x] Installation: Does installation proceed as outlined in the documentation?
• [x] Functionality: Have the functional claims of the software been confirmed?
• [x] Performance: If there are any performance claims of the software, have they been confirmed? (If there are no claims, please check off this item.)

Documentation

• [x] A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
• [x] Installation instructions: Is there a clearly-stated list of dependencies? Ideally these should be handled with an automated package management solution.
• [x] Example usage: Do the authors include examples of how to use the software (ideally to solve real-world analysis problems).
• [x] Functionality documentation: Is the core functionality of the software documented to a satisfactory level (e.g., API method documentation)?
• [x] Automated tests: Are there automated tests or manual steps described so that the functionality of the software can be verified?
• [x] Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support

Software paper

• [x] Summary: Has a clear description of the high-level functionality and purpose of the software for a diverse, non-specialist audience been provided?
• [x] A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
• [x] State of the field: Do the authors describe how this software compares to other commonly-used packages?
• [x] Quality of writing: Is the paper well written (i.e., it does not require editing for structure, language, or writing quality)?
• [x] References: Is the list of references complete, and is everything cited appropriately that should be cited (e.g., papers, datasets, software)? Do references in the text use the proper citation syntax?

[whedon]

Hello human, I'm @whedon, a robot that can help you with some common editorial tasks. @juliohm, @matbesancon it looks like you're currently assigned to review this paper :tada:.

:star: Important :star:

If you haven't already, you should seriously consider unsubscribing from GitHub notifications for this (https://github.com/openjournals/joss-reviews) repository. As a reviewer, you're probably currently watching this repository which means for GitHub's default behaviour you will receive notifications (emails) for all reviews 😿

To fix this do the following two things:

1. Set yourself as 'Not watching' https://github.com/openjournals/joss-reviews:

1. You may also like to change your default settings for this watching repositories in your GitHub profile here: https://github.com/settings/notifications

For a list of things I can do to help you, just type:

@whedon commands

For example, to regenerate the paper pdf after making changes in the paper's md or bib files, type:

@whedon generate pdf

[whedon]

Attempting to check references...

[whedon]

Attempting PDF compilation. Reticulating splines etc...

[whedon]

```Reference check summary:

OK DOIs

• 10.1287/mnsc.47.2.295.9834 is OK
• 10.1007/s10479-015-1994-2 is OK
• 10.1007/s101070000202 is OK
• 10.1137/080718401 is OK
• 10.1137/110825054 is OK
• 10.1007/s10589-015-9758-0 is OK
• 10.1137/15M1043376 is OK
• 10.1137/141000671 is OK
• 10.1016/j.ejor.2016.11.028 is OK
• https://doi.org/10.1016/0167-7152(92)90006-Q is OK

MISSING DOIs

• None

INVALID DOIs

• None
  ```

[whedon]

:point_right: Check article proof :page_facing_up: :point_left:

[kirui93]

@whedon generate pdf

[whedon]

Attempting PDF compilation. Reticulating splines etc...

[whedon]

:point_right: Check article proof :page_facing_up: :point_left:

[kirui93]

@whedon generate pdf

[whedon]

Attempting PDF compilation. Reticulating splines etc...

[whedon]

:point_right: Check article proof :page_facing_up: :point_left:

[melissawm]

Hi all, just checking in to see if there's anything I can help you with.

[matbesancon]

@melissawm a bunch of issues were opened, some being "nice-to-have" improvements on the package, but some being fairly necessary to navigate through the package and understand what is going on, how to use it & for which classes of problems. My opinion is that there is a fair bit of polishing still to do but I'll let the author voice in

[matbesancon]

cc @kirui93

[kirui93]

@matbesancon Thank you so much for mentioning that. I think I have implemented all the review comments that you had stated and that has enabled us to bump up the package. If you update to the new version, you will see that almost all the details are incorporated. What we are remaining with only is to change to use Plots.jl instead of PyPlot.jl which we will do it in some time to come as for now PyPlot.jl works well.

[kirui93]

@matbesancon I also did polish the paper and I think it is much of straightforward and to the point now.

[kirui93]

@matbesancon @juliohm I really believe that everything is now up to standard of the package we wanted to create. Your insightful review comments have been of great help.

[juliohm]

@kirui93 please comment on each open issue to explain how you addressed them. There are many open issues still.

[kirui93]

@juliohm I think most of the issues I have dealt with. So I will comment and close them if it is up to my side.

[kirui93]

@matbesancon @juliohm I did comment on the issues created and closed most of them as I had already dealt with them.

[juliohm]

@kirui93 you didn't address many issues raised in the software paper for example. You also didn't mention how you addressed each issue on GitHub before closing them.

Please report on each issue above how you addressed the review comments, otherwise it is difficult to approve the paper for publication in JOSS.

[kirui93]

@kirui93 you didn't address many issues raised in the software paper for example. You also didn't mention how you addressed each issue on GitHub before closing them.

Please report on each issue above how you addressed the review comments, otherwise it is difficult to approve the paper for publication in JOSS.

1. General comments about the documentation - Initially, I had my docstrings placed in a wrong manner and with the help of @matbesancon I was able to rewrite the docstrings in the correct manner. And also @juliohm had mentioned briefly about the usage of "library" instead of "package" in most parts of the documentation, which I addressed accordingly.

2. @matbesancon indicated about the title and the project set up. He was suggesting to name the package from "ScenTrees.jl" to something like "ScenarioTrees.jl". Initially, we had a lot of suggestions for the name of the package but we decided to go with "ScenTrees.jl" as it is unique and also in a way describes to our goal.

3. On naming and conventions, @matbesancon indicated about the function Children{Vector{Int64}} which looked like a constructor. As indicated, this function is an internal function of Tree and its main purpose is to reduce the number of times we are going to use the function in the stochastic approximation process. We access the children of the nodes each time in stochastic approximation iteration and hence saving the children as a vector at first with the "Chidren" function provides an easier and less expensive manner.

4. Another issue concerned incorporating AbstractTrees.jl. Though it was not a major issue, but as I had indicated, this package seems it is no longer maintained. And I couldn't find the documentation to detail its usage.

5. One of the most important issue that @matbesancon had stated was every function which performs a random sampling should take in parameter the Random Number Generator (which is usually a subtype of Random.AbstractRNG). I found this as a good first issue as as he had indicated making function runs deterministic makes it really important. With this I used the package Random.jl. As stated in their documentation, random number generation in Julia uses the Mersenne Twister library via MersenneTwister objects and thus I employed this for every function that involves random number generation.

6. There was quite a mislead regarding the Wasserstein distance and we thank @matbesancon for noting this out. In our code, we had used the term "WassersteinDistance" which was inappropriate. I stated in the issue that this was supposed to be the multistage distance and is duly corrected in the code. There is also the parameter r which is used for multistage distance and is also corrected well in all the codes.

7. The final issue that @juliohm opened was about improvements for the software paper. All the corrections he suggested are duly represented in the paper. The current version of the paper has all the corrections implemented.

Thank you so much @matbesancon and @juliohm for your patience as well as insightful comments about the software paper as well as the package in general.

[juliohm]

@kirui93 can you please add these comments to the issues in your project repo instead? It is quite hard to follow any text here without context. Please understand that we have limited time to review.

I will try to read the issues again in the following weeks.

Thank you.

[juliohm]

@whedon generate pdf

[whedon]

Attempting PDF compilation. Reticulating splines etc...

[whedon]

:point_right: Check article proof :page_facing_up: :point_left:

[kirui93]

@juliohm @matbesancon Thank you so much for your patience. I have re-commented again on the important issues and re-opened them.

[kirui93]

Some other issues are general comments that were just suggestions for improving the documentation as well as docstrings which I did implement immediately on statement.

[kirui93]

@whedon generate pdf

[whedon]

Attempting PDF compilation. Reticulating splines etc...

[whedon]

:point_right: Check article proof :page_facing_up: :point_left:

[melissawm]

Hello all, just checking in on this review, is there anything I can help you with?

[matbesancon]

still a few code-related issues we are discussing with the author on my side

[juliohm]

Thank you for checking the status of the submission Melissa. The author
didn't address all the issues raised. I can take a look again if any update
is triggered.

On Fri, Jan 10, 2020, 09:29 Mathieu Besançon [e-mail]
wrote:

still a few code-related issues we are discussing with the author on my
side

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/openjournals/joss-reviews/issues/1912?email_source=notifications&email_token=AAZQW3IDEH56V46DBSZXZRTQ5BS3FA5CNFSM4JQQHSFKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEITYXBQ#issuecomment-573016966,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAZQW3OKQ75NLTGVW75IG3LQ5BS3FANCNFSM4JQQHSFA
.

[kirui93]

Hello @juliohm, I addressed all the issues that you had raised concerning the software paper. All the other issues also that were raised concerning the package are also addressed. You had mentioned some times back that I should comment on all the issues raised on how I addressed them and I did. Please have a look on my comments on the way I addressed the issues.

[kirui93]

Hello @matbesancon, I gave you a reply on what we were threading on last time. Please have a look on it.

[kirui93]

Hi @melissawm, thank you for reaching out again

[juliohm]

Thank you @kirui93 for the update. On my side you are ready to go, let's wait for @matbesancon 's comments, he may have further comments to improve the work.

[kirui93]

@juliohm Thank you so much for your insightful comments as well as your time to review our package and the paper.

[kirui93]

@whedon generate pdf

[whedon]

:point_right: Check article proof :page_facing_up: :point_left:

[matbesancon]

As an update, only https://github.com/kirui93/ScenTrees.jl/issues/15 remains on the documentation side, I'll do a last pass on the paper but it starts to all look good to me, @juliohm do you have something more?

[juliohm]

Everything is good to go on my side @matbesancon , thanks for pinging.

[kirui93]

@whedon generate pdf

[whedon]

:point_right: Check article proof :page_facing_up: :point_left:

[matbesancon]

@whedon check references

[whedon]

Reference check summary:

OK DOIs

- 10.1287/mnsc.47.2.295.9834 is OK
- 10.1007/s10479-015-1994-2 is OK
- 10.1007/s101070000202 is OK
- 10.1137/080718401 is OK
- 10.1137/110825054 is OK
- 10.1007/s10589-015-9758-0 is OK
- 10.1137/15M1043376 is OK
- 10.1137/141000671 is OK
- 10.1016/j.ejor.2016.11.028 is OK

MISSING DOIs

- None

INVALID DOIs

- https://doi.org/10.1016/0167-7152(92)90006-Q is INVALID because of 'https://doi.org/' prefix

[matbesancon]

@kirui93 can you check the DOI above, there is one that needs removing the https://doi.org prefix

[kirui93]

Yeah. It was a slight mistake in the .bib file. I have removed it.

[matbesancon]

@whedon check references

[whedon]

Reference check summary:

OK DOIs

- 10.1287/mnsc.47.2.295.9834 is OK
- 10.1007/s10479-015-1994-2 is OK
- 10.1007/s101070000202 is OK
- 10.1137/080718401 is OK
- 10.1137/110825054 is OK
- 10.1007/s10589-015-9758-0 is OK
- 10.1137/15M1043376 is OK
- 10.1137/141000671 is OK
- 10.1016/j.ejor.2016.11.028 is OK

MISSING DOIs

- None

INVALID DOIs

- https://doi.org/10.1016/0167-7152(92)90006-Q is INVALID because of 'https://doi.org/' prefix

[matbesancon]

problem is still there

[kirui93]

I have removed it. Please have a look again.

[matbesancon]

@whedon check references

[whedon]

Reference check summary:

OK DOIs

- 10.1287/mnsc.47.2.295.9834 is OK
- 10.1007/s10479-015-1994-2 is OK
- 10.1007/s101070000202 is OK
- 10.1137/080718401 is OK
- 10.1137/110825054 is OK
- 10.1007/s10589-015-9758-0 is OK
- 10.1137/15M1043376 is OK
- 10.1137/141000671 is OK
- 10.1016/j.ejor.2016.11.028 is OK

MISSING DOIs

- None

INVALID DOIs

- https://doi.org/10.1016/0167-7152(92)90006-Q is INVALID because of 'https://doi.org/' prefix

[kirui93]

I have removed this reference in paper.pip file because I didn't even cite it. I am wondering why whedon is saying invalid to a reference already removed

[kirui93]

@whedon generate pdf

[whedon]

:point_right: Check article proof :page_facing_up: :point_left:

[matbesancon]

@whedon check references

[whedon]

Reference check summary:

OK DOIs

- 10.1287/mnsc.47.2.295.9834 is OK
- 10.1007/s10479-015-1994-2 is OK
- 10.1007/s101070000202 is OK
- 10.1137/080718401 is OK
- 10.1137/110825054 is OK
- 10.1007/s10589-015-9758-0 is OK
- 10.1137/15M1043376 is OK
- 10.1137/141000671 is OK
- 10.1016/j.ejor.2016.11.028 is OK

MISSING DOIs

- None

INVALID DOIs

- None

[matbesancon]

@melissawm it all seems good to me now, I just ticked the last cases

[melissawm]

Thank you both, @matbesancon and @juliohm !

[melissawm]

@kirui93 before we finish, can I ask you to look into some details?

• In the first paragraph of the summary, the word "generation" in "scenario generation" is broken; can you fix this?
• In the reference Høyland, K., & Wallace, S. W. (2001) there is a typo in the word Science in "Management Sceince"
• In the reference Kovacevic, R. M., & Pichler, A. (2015), I believe you're missing a 235 as the citation seems to be "Kovacevic, R.M., Pichler, A. Tree approximation for discrete time stochastic processes: a process distance approach. Ann Oper Res 235, 395–421 (2015). https://doi.org/10.1007/s10479-015-1994-2"

After those I'll ask you to generate a new proof and double check so we can generate the final archive. Thanks!

[kirui93]

@melissawm Thanks for the comments.

• Is there a way to prevent a word break really in a markdown document?

[kirui93]

@whedon generate pdf

[whedon]

:point_right: Check article proof :page_facing_up: :point_left:

[kirui93]

@melissawm Thanks for the comments.

• Is there a way to prevent a word break really in a markdown document?

@melissawm I think I have got it.
I have corrected everything.

[melissawm]

Hello, @kirui93 . Can you make a Zenodo archive, and report the DOI so we can set the final archive? Thank you

[kirui93]

Hello @melissawm

The DOI for the Zenodo archive is

https://doi.org/10.5281/zenodo.3672205

The version adapted to the DOI is the latest version v0.2.2

Thank you.

[melissawm]

Hi @kirui93 please check the metadata on the Zenodo archive, especially the title and authors list, as the citation reads "Kipngeno Kirui, Júlio Hoffimann, & Julia TagBot. (2020, February 18). kirui93/ScenTrees.jl: Release for zenodo (Version v0.2.2). Zenodo. http://doi.org/10.5281/zenodo.3672205"

[kirui93]

Hello @melissawm,
At first I had confused how to upload to Zenodo. Here is the same DOI

http://doi.org/10.5281/zenodo.3672205

The version adapted is still v0.2.2

Thank you for your patience.

[melissawm]

@whedon set http://doi.org/10.5281/zenodo.3672205 as archive

[whedon]

OK. 10.5281/zenodo.3672205 is the archive.

[melissawm]

@whedon set v0.2.2 as version

[whedon]

OK. v0.2.2 is the version.

[melissawm]

@openjournals/joss-eics I think this is ready!

Thanks @kirui93 and reviewers @matbesancon and @juliohm !

[kirui93]

Thanks so much @melissawm for your patience.
Thank you so much to @matbesancon and @juliohm for your important and well informed reviews. I appreciate your time and your efforts to review this submission.

[kyleniemeyer]

Everything looks good to me, so I'll proceed with acceptance.

[kyleniemeyer]

@whedon accept

[whedon]

Attempting dry run of processing paper acceptance...

[whedon]

Reference check summary:

OK DOIs

- 10.1287/mnsc.47.2.295.9834 is OK
- 10.1007/s10479-015-1994-2 is OK
- 10.1007/s101070000202 is OK
- 10.1137/080718401 is OK
- 10.1137/110825054 is OK
- 10.1007/s10589-015-9758-0 is OK
- 10.1137/15M1043376 is OK
- 10.1137/141000671 is OK
- 10.1016/j.ejor.2016.11.028 is OK

MISSING DOIs

- None

INVALID DOIs

- None

[whedon]

Check final proof :point_right: https://github.com/openjournals/joss-papers/pull/1324

If the paper PDF and Crossref deposit XML look good in https://github.com/openjournals/joss-papers/pull/1324, then you can now move forward with accepting the submission by compiling again with the flag deposit=true e.g.
@whedon accept deposit=true

[kyleniemeyer]

@whedon accept deposit=true

[whedon]

Doing it live! Attempting automated processing of paper acceptance...

[whedon]

🐦🐦🐦 👉 Tweet for this paper 👈 🐦🐦🐦

[whedon]

🚨🚨🚨 THIS IS NOT A DRILL, YOU HAVE JUST ACCEPTED A PAPER INTO JOSS! 🚨🚨🚨

Here's what you must now do:

1. Check final PDF and Crossref metadata that was deposited :point_right: https://github.com/openjournals/joss-papers/pull/1325
2. Wait a couple of minutes to verify that the paper DOI resolves https://doi.org/10.21105/joss.01912
3. If everything looks good, then close this review issue.

4. Party like you just published a paper! 🎉🌈🦄💃👻🤘

   Any issues? notify your editorial technical team...

[kyleniemeyer]

Congrats @kirui93 on your article's publication in JOSS!

Many thanks to @juliohm and @matbesancon for reviewing this, and @melissawm for editing.

[whedon]

:tada::tada::tada: Congratulations on your paper acceptance! :tada::tada::tada:

If you would like to include a link to your paper from your README use the following code snippets:

Markdown:
[![DOI](https://joss.theoj.org/papers/10.21105/joss.01912/status.svg)](https://doi.org/10.21105/joss.01912)

HTML:
<a style="border-width:0" href="https://doi.org/10.21105/joss.01912">
  <img src="https://joss.theoj.org/papers/10.21105/joss.01912/status.svg" alt="DOI badge" >
</a>

reStructuredText:
.. image:: https://joss.theoj.org/papers/10.21105/joss.01912/status.svg
   :target: https://doi.org/10.21105/joss.01912

This is how it will look in your documentation:

We need your help!

Journal of Open Source Software is a community-run journal and relies upon volunteer effort. If you'd like to support us please consider doing either one (or both) of the the following:

• Volunteering to review for us sometime in the future. You can add your name to the reviewer list here: https://joss.theoj.org/reviewer-signup.html
• Making a small donation to support our running costs here: https://numfocus.org/donate-to-joss