Joss-reviews: [PRE REVIEW]: CIMSA: Customizable Integrated Multiple Sequence Alignment Package

[whedon]

Submitting author: @kuangmeng (Mengmeng Kuang)
Repository: https://github.com/kuangmeng/CIMSA
Version: v1.0
Editor: @will-rowe
Reviewers: @CFGrote, @xyzmjf
Managing EiC: Lorena A Barba

:warning: JOSS reduced service mode :warning:

Due to the challenges of the COVID-19 pandemic, JOSS is currently operating in a "reduced service mode". You can read more about what that means in our blog post.

Author instructions

Thanks for submitting your paper to JOSS @kuangmeng. Currently, there isn't an JOSS editor assigned to your paper.

@kuangmeng if you have any suggestions for potential reviewers then please mention them here in this thread (without tagging them with an @). In addition, this list of people have already agreed to review for JOSS and may be suitable for this submission (please start at the bottom of the list).

Editor instructions

The JOSS submission bot @whedon is here to help you find and assign reviewers and start the main review. To find out what @whedon can do for you type:

@whedon commands

[whedon]

Hello human, I'm @whedon, a robot that can help you with some common editorial tasks.

:warning: JOSS reduced service mode :warning:

Due to the challenges of the COVID-19 pandemic, JOSS is currently operating in a "reduced service mode". You can read more about what that means in our blog post.

For a list of things I can do to help you, just type:

@whedon commands

For example, to regenerate the paper pdf after making changes in the paper's md or bib files, type:

@whedon generate pdf

[whedon]

Failed to discover a Statement of need section in paper

[whedon]

Software report (experimental):

github.com/AlDanial/cloc v 1.84  T=0.41 s (68.0 files/s, 20563.8 lines/s)
-------------------------------------------------------------------------------
Language                     files          blank        comment           code
-------------------------------------------------------------------------------
C++                              5            475            430           2787
C/C++ Header                    13            579            947           2716
Markdown                         2             51              0            131
Python                           2             32             10             96
TeX                              1             10              0             88
HTML                             1              0              1             53
JavaScript                       2              1              8             28
make                             1              5              0             12
Bourne Shell                     1              1              0             10
-------------------------------------------------------------------------------
SUM:                            28           1154           1396           5921
-------------------------------------------------------------------------------


Statistical information for the repository '2493' was gathered on 2020/07/18.
The following historical commit information, by author, was found:

Author                     Commits    Insertions      Deletions    % of changes
kuangmeng                        7          8386            277          100.00

Below are the number of rows from each author that have survived and are still
intact in the current revision:

Author                     Rows      Stability          Age       % in comments
Mengmeng Kuang             8109          100.0          0.0               18.19

[whedon]

Failed to discover a valid open source license.

[whedon]

:point_right: Check article proof :page_facing_up: :point_left:

[whedon]

Reference check summary:

OK DOIs

- None

MISSING DOIs

- https://doi.org/10.1371/journal.pcbi.0030123 may be missing for title: Recent evolutions of multiple sequence alignment algorithms
- https://doi.org/10.1093/bioinformatics/btl472 may be missing for title: Probalign: multiple sequence alignment using partition function posterior probabilities
- https://doi.org/10.1093/bioinformatics/btq338 may be missing for title: MSAProbs: multiple sequence alignment based on pair hidden Markov models and partition function posterior probabilities
- https://doi.org/10.1109/bibm.2018.8621220 may be missing for title: ProbPFP: a multiple sequence alignment algorithm combining partition function and hidden markov model with particle swarm optimization
- https://doi.org/10.1093/comjnl/16.1.30 may be missing for title: SLINK: an optimally efficient algorithm for the single-link cluster method
- https://doi.org/10.1093/comjnl/20.4.364 may be missing for title: An efficient algorithm for a complete link method

INVALID DOIs

- None

[labarba]

👋 @kuangmeng — I'm going to send this back for a bit of work on the paper. Currently, it does not follow our guidance in providing a clear _Statement of Need_. See the section What should my paper contain? in our documentation. Tell us why the research world needs your software!

Also, to be eligible for consideration in JOSS, your software must be under a standard public license, approved by the Open Source Initiative. See our documentation.

[labarba]

@will-rowe — would you have a look at this submission and let me know if you might be able to handle it as editor?

[kuangmeng]

@whedon generate pdf

[whedon]

:point_right: Check article proof :page_facing_up: :point_left:

[kuangmeng]

@labarba The DOI issue, the open source license issue, and the Statement of Need section issue have been fixed. Thanks.

[kuangmeng]

@whedon commands

[whedon]

Here are some things you can ask me to do:

# List Whedon's capabilities
@whedon commands

# List of editor GitHub usernames
@whedon list editors

# List of reviewers together with programming language preferences and domain expertise
@whedon list reviewers

EDITORIAL TASKS

# Compile the paper
@whedon generate pdf

# Compile the paper from alternative branch
@whedon generate pdf from branch custom-branch-name

# Ask Whedon to check the references for missing DOIs
@whedon check references

# Ask Whedon to check repository statistics for the submitted software
@whedon check repository

[kuangmeng]

@whedon check references

[kuangmeng]

@whedon check repository

[whedon]

Reference check summary:

OK DOIs

- 10.1101/gr.2821705 is OK
- 10.1016/j.sbi.2006.04.004 is OK
- 10.1371/journal.pcbi.0030123 is OK
- 10.1093/bioinformatics/btl472 is OK
- 10.1093/bioinformatics/btq338 is OK
- 10.1109/TCBB.2014.2316820 is OK
- 10.1109/bibm.2018.8621220 is OK
- 10.1093/comjnl/16.1.30 is OK
- 10.1093/comjnl/20.4.364 is OK

MISSING DOIs

- None

INVALID DOIs

- None

[whedon]

Software report (experimental):

github.com/AlDanial/cloc v 1.84  T=0.76 s (37.0 files/s, 11198.8 lines/s)
-------------------------------------------------------------------------------
Language                     files          blank        comment           code
-------------------------------------------------------------------------------
C++                              5            475            430           2787
C/C++ Header                    13            579            947           2716
Markdown                         2             51              0            131
TeX                              1             10              0             97
Python                           2             32             10             96
HTML                             1              0              1             53
JavaScript                       2              1              8             28
make                             1              5              0             12
Bourne Shell                     1              1              0             10
-------------------------------------------------------------------------------
SUM:                            28           1154           1396           5930
-------------------------------------------------------------------------------


Statistical information for the repository '2493' was gathered on 2020/07/18.
The following historical commit information, by author, was found:

Author                     Commits    Insertions      Deletions    % of changes
kuangmeng                        7          8386            277          100.00

Below are the number of rows from each author that have survived and are still
intact in the current revision:

Author                     Rows      Stability          Age       % in comments
Mengmeng Kuang             8109          100.0          0.0               18.19

[whedon]

:point_right: Check article proof :page_facing_up: :point_left:

[kuangmeng]

@whedon generate pdf

[whedon]

:point_right: Check article proof :page_facing_up: :point_left:

[will-rowe]

@labarba - I'm on leave for the week but happy to handle when I get back

[labarba]

@whedon assign @will-rowe as editor

[whedon]

OK, the editor is @will-rowe

[will-rowe]

Hi @kuangmeng

Thanks for the submission! I've been reading through it this morning and have a couple of things I'd like before we put this out for review.

• I'm not an MSA algorithm researcher so am finding this a little hard to find a use case for. It would be great if you could provide a usage example, ideally for a real-world analysis problem. This will also help me find reviewers for your submission. This would be really beneficial at the top of the README.

• Please could you add some additional testing. I see that you have a test to make sure the compiled binary works with all available parameters, this is good! But I think we really need some unit testing - particularly where you have reimplemented MSA algorithms. It would be great if you could also hook this up to continuous integration (e.g. travis)

• I think the documentation needs a little work beyond what you have in the README. It would be really useful to have some more information on the various parameters, what the methods you have reimplemented are doing and how to access/interpret the results. Your figure from the paper would be useful here too. To save the README getting too crowded, you could add documentation via GitHub wiki or a service like readthedocs.

Sorry that these are quite significant points, but I really think they need doing before we can review this. Let me know if you need any further clarification or help.

[kuangmeng]

@will-rowe Thanks for your comments and I will revise the README and additional testing issues ASAP.

[kuangmeng]

Hi @will-rowe

Thanks for your comments and I have revised the README file for your 1st and 3rd comments. As for 2nd comment, I will try my best to do it.

[kuangmeng]

Dear @will-rowe ,

Thank you for waiting so long. Since the calculation results of each step of my CIMSA are uncertain, I can't add normal unit tests. Therefore, in order to determine the correctness of each step, I add the assert() statement and print the intermediate running state. I hope this can help you understand my program better. Thanks.

[kuangmeng]

@labarba @will-rowe Please let me know what I should provide for further consideration. I really appreciate your comments and suggestions. Thanks.

[will-rowe]

Hi @kuangmeng - thanks for your update. My concern is that this still falls short of our reviewer criteria:

No way for you, the reviewer, to objectively assess whether the software works

This might be my fault for not being specific enough. Could you include some sort of "truth set". For example, a set of sequences and the expected MSA? These expected MSAs could be generated using other tools (maybe clustal for instance?). A minimal example could be to have a test that takes an input set, produces an MSA using your software and then diffs this against a reference MSA.

Here is an example I just found of another MSA aligner doing this. Personally, I'd prefer it done using a testing framework, and for fail/edge cases to be considered, but that is not a requirement for review (but might make your life easier when maintaining the software :))

[kuangmeng]

Thanks for your suggestions @will-rowe and I have added the test_cimsa.sh to test whether CIMSA can work on an example. And I have tested it successfully.

[will-rowe]

Thanks @kuangmeng - I think this meets are criteria for review now. Let's find some reviewers and then start the process.

[will-rowe]

Hi @CFGrote and @VivekTodur - would you both be willing to review this submission please?

[kuangmeng]

Thanks @kuangmeng - I think this meets are criteria for review now. Let's find some reviewers and then start the process.

Many thanks. @will-rowe

[CFGrote]

Thanks for the offer, I accept.

[will-rowe]

Thanks @CFGrote - that's brilliant. I'll track down one more reviewer and then we'll start

[will-rowe]

Hi @xyzmjf and @billchenxi - are either of you please able to review this submission for JOSS?

[will-rowe]

Hi @bblum9, @lmvasquezg and @emilydolson - would any of you be able to review this submission for JOSS please?

[xyzmjf]

Hi Will

Apologies for taking a few days to reply. I can review this manuscript.
What timeline are you expecting, and what format would you want the review
in.
As this is the first time I will have reviewed for you, I do have a few
'newbie' questions.

Mark

On Mon, Sep 14, 2020 at 9:33 AM Will Rowe [e-mail] wrote:

Hi @bblum9 https://github.com/bblum9, @lmvasquezg
https://github.com/lmvasquezg and @emilydolson
https://github.com/emilydolson - would any of you be able to review
this submission for JOSS please?

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/openjournals/joss-reviews/issues/2493#issuecomment-691906798,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AFRJKDF4ALRSW72VFTHIQDLSFXINLANCNFSM4PAFMT2A
.

[bblum9]

I'm also available to review this submission.

[will-rowe]

Perfect - thanks @xyzmjf

Please find the full reviewer documentation here.

We will start a new thread in a moment which will record the submission process and this one will be closed. You can use the new thread to track the review process, mark off your reviewer checklist and let us know your recommendation etc. As well as this, we suggest writing your reviewer comments and feedback as a Github issue on the submitting author's repository (https://github.com/kuangmeng/CIMSA).

I'll start the review thread in a moment and please feel free to ask any question there. We are pretty flexible on how long the review takes; usually it's best if it can be received within a few weeks but I appreciate that timelines are tricky for everyone at the moment.

[will-rowe]

@bblum9 - thank you for your quick response but we now have 2 reviewers. I really appreciate you getting back to me and I hope you will be available to review for us in the future

[will-rowe]

@whedon assign @CFGrote as reviewer

[whedon]

OK, @CFGrote is now a reviewer

[will-rowe]

@whedon add @xyzmjf as reviewer

[whedon]

OK, @xyzmjf is now a reviewer

[will-rowe]

@whedon start review

[whedon]

OK, I've started the review over in https://github.com/openjournals/joss-reviews/issues/2672.