Joss-reviews: [REVIEW]: mpi4py-fft - Parallel Fast Fourier Transforms with MPI for Python

[whedon]

Submitting author: @mikaem (Mikael Mortensen)
Repository: https://bitbucket.org/mpi4py/mpi4py-fft
Version: 2.0.2
Editor: @VivianePons
Reviewer: @iljah, @rainwoodman
Archive: 10.5281/zenodo.2621442

Status

Status badge code:

HTML: <a href="http://joss.theoj.org/papers/01fab321206ae08b5dbfb0ccb7967f78"><img src="http://joss.theoj.org/papers/01fab321206ae08b5dbfb0ccb7967f78/status.svg"></a>
Markdown: [![status](http://joss.theoj.org/papers/01fab321206ae08b5dbfb0ccb7967f78/status.svg)](http://joss.theoj.org/papers/01fab321206ae08b5dbfb0ccb7967f78)

Reviewers and authors:

Please avoid lengthy details of difficulties in the review thread. Instead, please create a new issue in the target repository and link to those issues (especially acceptance-blockers) in the review thread below. (For completists: if the target issue tracker is also on GitHub, linking the review thread in the issue or vice versa will create corresponding breadcrumb trails in the link target.)

Reviewer instructions & questions

@iljah & @rainwoodman, please carry out your review in this issue by updating the checklist below. If you cannot edit the checklist please:

1. Make sure you're logged in to your GitHub account
2. Be sure to accept the invite at this URL: https://github.com/openjournals/joss-reviews/invitations

The reviewer guidelines are available here: https://joss.theoj.org/about#reviewer_guidelines. Any questions/concerns please let @VivianePons know.

✨ Please try and complete your review in the next two weeks ✨

Review checklist for @iljah

Conflict of interest

• [x] As the reviewer I confirm that I have read the JOSS conflict of interest policy and that there are no conflicts of interest for me to review this work.

Code of Conduct

• [x] I confirm that I read and will adhere to the JOSS code of conduct.

General checks

• [x] Repository: Is the source code for this software available at the repository url?
• [x] License: Does the repository contain a plain-text LICENSE file with the contents of an OSI approved software license?
• [x] Version: 2.0.2
• [x] Authorship: Has the submitting author (@mikaem) made major contributions to the software? Does the full list of paper authors seem appropriate and complete?

Functionality

• [x] Installation: Does installation proceed as outlined in the documentation?
• [x] Functionality: Have the functional claims of the software been confirmed?
• [x] Performance: If there are any performance claims of the software, have they been confirmed? (If there are no claims, please check off this item.)

Documentation

• [x] A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
• [x] Installation instructions: Is there a clearly-stated list of dependencies? Ideally these should be handled with an automated package management solution.
• [x] Example usage: Do the authors include examples of how to use the software (ideally to solve real-world analysis problems).
• [x] Functionality documentation: Is the core functionality of the software documented to a satisfactory level (e.g., API method documentation)?
• [x] Automated tests: Are there automated tests or manual steps described so that the function of the software can be verified?
• [x] Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support

Software paper

• [x] Authors: Does the paper.md file include a list of authors with their affiliations?
• [x] A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
• [x] References: Do all archival references that should have a DOI list one (e.g., papers, datasets, software)?

Review checklist for @rainwoodman

Conflict of interest

• [x] As the reviewer I confirm that I have read the JOSS conflict of interest policy and that there are no conflicts of interest for me to review this work.

Code of Conduct

• [x] I confirm that I read and will adhere to the JOSS code of conduct.

General checks

• [x] Repository: Is the source code for this software available at the repository url?
• [x] License: Does the repository contain a plain-text LICENSE file with the contents of an OSI approved software license?
• [x] Version: 2.0.2
• [x] Authorship: Has the submitting author (@mikaem) made major contributions to the software? Does the full list of paper authors seem appropriate and complete?

Functionality

• [x] Installation: Does installation proceed as outlined in the documentation?
• [x] Functionality: Have the functional claims of the software been confirmed?
• [x] Performance: If there are any performance claims of the software, have they been confirmed? (If there are no claims, please check off this item.)

Documentation

• [x] A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
• [x] Installation instructions: Is there a clearly-stated list of dependencies? Ideally these should be handled with an automated package management solution.
• [x] Example usage: Do the authors include examples of how to use the software (ideally to solve real-world analysis problems).
• [x] Functionality documentation: Is the core functionality of the software documented to a satisfactory level (e.g., API method documentation)?
• [x] Automated tests: Are there automated tests or manual steps described so that the function of the software can be verified?
• [x] Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support

Software paper

• [x] Authors: Does the paper.md file include a list of authors with their affiliations?
• [x] A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
• [x] References: Do all archival references that should have a DOI list one (e.g., papers, datasets, software)?

[whedon]

Hello human, I'm @whedon, a robot that can help you with some common editorial tasks. @iljah, it looks like you're currently assigned as the reviewer for this paper :tada:.

:star: Important :star:

If you haven't already, you should seriously consider unsubscribing from GitHub notifications for this (https://github.com/openjournals/joss-reviews) repository. As a reviewer, you're probably currently watching this repository which means for GitHub's default behaviour you will receive notifications (emails) for all reviews 😿

To fix this do the following two things:

1. Set yourself as 'Not watching' https://github.com/openjournals/joss-reviews:

1. You may also like to change your default settings for this watching repositories in your GitHub profile here: https://github.com/settings/notifications

For a list of things I can do to help you, just type:

@whedon commands

[whedon]

Attempting PDF compilation. Reticulating splines etc...

[whedon]

:point_right: Check article proof :page_facing_up: :point_left:

[iljah]

@VivianePons bitbucket doesn't seem to have releases so this item isn't applicable "Does the release version given match the GitHub release (v2.0.0)". I'll mark it as completed...

[mikaem]

@iljah The most recent version released on pypi and conda-forge is now 2.0.1. I need to change the JOSS version number. The tags on bitbucket, used for these releases, are here.

[iljah]

@mikaem this condition might not hold for mpi4py-fft: "Does the repository contain a plain-text LICENSE file". Even though rst isn't a binary format it still doesn't look like plain text when viewed in a plain text editor.

[iljah]

I couldn't find examples on how to use mpi4py-fft as required by "Example usage: Do the authors include examples of how to use the software". At least one simple example would be informative.

[iljah]

Community guidelines as listed in our checklists also seem to be missing.

[iljah]

Functionality documentation also seems to be missing, unless using external websites is allowed, @VivianePons ?

[iljah]

Ah looks like most documentation is "hidden" in docs/source directory. Mentioning this directory in top level readme should be sufficient to cover doc requirements...

[dalcinl]

@iljah About LICENSE.rst: In general lines, reStructuredText, Markdown, etc can be considered plain-text, they were designed with very lightweight markup in mind. In the very particular case of our LICENSE.rst, we only deviate for extreme pure-text in the two fields :Author: and :Contact:, however this markup is extremely lightweight, IMHO it does not interfere at all with reading it in a text editor, and we get the bonus of nice rendering when viewed in Bitbucket or GitHub.

Of course, I'm not a lawyer. @VivianePons What should we do?

@mikaem Anyway, we need to update the license file to read 2017-2019.

[dalcinl]

@iljah The documentation is not "hidden" in docs/source. It is just the reStructuredText sources for what is available online in https://mpi4py-fft.readthedocs.io. BTW, our README file has the usual docs:passing badge at the top with the link to readthedocs.

[dalcinl]

@iljah About examples, did you take a look under the examples directory?

[arfon]

Of course, I'm not a lawyer. @VivianePons What should we do?

IANAL too but I think this is OK.

[iljah]

our README file has the usual docs:passing badge at the top with the link to readthedocs

As I understood my instructions, documentation should be in readme file so link to external website isn't sufficient, @VivianePons ? I think just cloning the repo should give everything my instructions are asking for, at least minimal versions of each item.

did you take a look under the examples directory?

No as it wasn't mentioned in readme. I guess examples dir is good enough but mentioning it would be nice.

[iljah]

I couldn't find separate API documentation (in git repo) but using dir and help commands on mpi4py_fft and its contents looks good enough.

[VivianePons]

@whedon set version 2.0.1

[VivianePons]

@whedon set 2.0.1 as version

[whedon]

OK. 2.0.1 is the version.

[VivianePons]

licence: I believe a .rst file is ok
documentation: the readthedoc version is ok, considering the source is in the repo. It would be good to have a link from the readme
Same for "examples", make it easy for a user to find everything from the readme file by just adding a few links.

[rainwoodman]

@VivianePons : Do I need to pay special attention to avoid biasing my review by seeing other reviewer's comments?

[VivianePons]

No don't worry about that, the idea is that the discussion is open between both reviewers and the authors. I trust your judgement anyway ;)

[rainwoodman]

The guideline calls for citations to existing software performing similar functionalities.
The submission contains citations that demonstrates the merit / usefulness of this software, but is missing a citations to existing software that also performs distributed parallel FFTs: fftw-mpi, p3dfft, pfft, accfft, to name a few.

[rainwoodman]

API: https://mpi4py-fft.readthedocs.io/en/latest/mpi4py_fft.html#mpi4py_fft.mpifft.PFFT
The main entrance of PFFT takes a boolean argument pyfftw. The use of boolean makes it less clean to add support to other 1d FFT backends.

[rainwoodman]

@VivianePons Is there a mechanism to submit comments in bulk and keep track of them?

[iljah]

This might work well: https://help.github.com/en/articles/about-task-lists

[mikaem]

The guideline calls for citations to existing software performing similar functionalities.
The submission contains citations that demonstrates the merit / usefulness of this software, but is missing a citations to existing software that also performs distributed parallel FFTs: fftw-mpi, p3dfft, pfft, accfft, to name a few.

@rainwoodman Thank you for the feedback. The guidelines (under What should my paper contain?) state that we should include key references and Mentions (if applicable) of any ongoing research projects using the software or recent scholarly publications enabled by it. There seem to be no call for citations to existing software performing similar functionalities.

[mikaem]

@VivianePons Thank you for the clarifications.

licence: I believe a .rst file is ok
documentation: the readthedoc version is ok, considering the source is in the repo. It would be good to have a link from the readme

@VivianePons There are already three links to the documentation from the landing cite on bitbucket. Two of these links are in the readme file.

Same for "examples", make it easy for a user to find everything from the readme file by just adding a few links.

The examples directory contains some code snippets that are not really well documented. There are plenty of examples in the documentation and we prefer to guide the users in that direction. For this reason we prefer, for now, not to link to the examples.

[iljah]

Since links to external sites are ok to me this seems good to go.

[VivianePons]

@rainwoodman I don't know anything specific except from mentioned checklist (as in the issue description). Do you have comments to give to the author?

[rainwoodman]

I got interrupted. I will wrap this up before Friday.

[rainwoodman]

A few additional comments:

1. I really liked the distarray object. However, for a user it feels awkward to import an FFT library to use a MPI distributed array; the implication is that the commitment on the support may be less than that of the PFFT objects, no matter what the intent from the authors is. Are there plans or roadmap items regarding this?

2. At a certain point we ran into a discussion about life cycle management of MPI communicators. Some implementations require collective destruction of MPI communicators. SubComm supports an explicit destroy method, probably as a way to address this. Some objects in the package owns Subcomm, and does not always invoke the destroy method. I recommend adding documentation on the need to invoke the destroy method on the SubComm objects by the users, otherwise there will be leaks.

3. This is an item required on the checklist. On my installation, running the test failed with

(base) [yfeng1@waterfall tests]$ bash runtests.sh
Traceback (most recent call last):
  File "/home/yfeng1/source/mpi4py-fft/tests/test_fftw.py", line 162, in <module>
    test_fftw()
  File "/home/yfeng1/source/mpi4py-fft/tests/test_fftw.py", line 105, in test_fftw
    assert allclose(A, A2), np.linalg.norm(A-A2)
AssertionError: 3.7629273
(base) [yfeng1@waterfall tests]$

On master branch and 2.0.1. Following instructions on
https://mpi4py-fft.readthedocs.io/en/latest/installation.html#test-installation

Admittedly I did not set up tests on a clean environment, but this is my production environment where most packages work out just fine.

[mikaem]

@rainwoodman Thank you for the comments. A few remarks are given below

A few additional comments:

1. I really liked the distarray object. However, for a user it feels awkward to import an FFT library to use a MPI distributed array; the implication is that the commitment on the support may be less than that of the PFFT objects, no matter what the intent from the authors is. Are there plans or roadmap items regarding this?

There are no plans or roadmaps regarding this. The distribution of a global array is an intricate part of the algorithm implemented in mpi4py-fft. We do not really aim for a generic distributed array package. Note that the distributed array requires at least one undivided axis, and this is suboptimal for most applications.

1. At a certain point we ran into a discussion about life cycle management of MPI communicators. Some implementations require collective destruction of MPI communicators. SubComm supports an explicit destroy method, probably as a way to address this. Some objects in the package owns Subcomm, and does not always invoke the destroy method. I recommend adding documentation on the need to invoke the destroy method on the SubComm objects by the users, otherwise there will be leaks.

That is a good idea.

1. This is an item required on the checklist. On my installation, running the test failed with

(base) [yfeng1@waterfall tests]$ bash runtests.sh
Traceback (most recent call last):
  File "/home/yfeng1/source/mpi4py-fft/tests/test_fftw.py", line 162, in <module>
    test_fftw()
  File "/home/yfeng1/source/mpi4py-fft/tests/test_fftw.py", line 105, in test_fftw
    assert allclose(A, A2), np.linalg.norm(A-A2)
AssertionError: 3.7629273
(base) [yfeng1@waterfall tests]$

On master branch and 2.0.1. Following instructions on
https://mpi4py-fft.readthedocs.io/en/latest/installation.html#test-installation

Admittedly I did not set up tests on a clean environment, but this is my production environment where most packages work out just fine.

This error does not seem to be reproducible. All tests pass on CircleCI and on the pipeline. Furthermore, the tests pass on my local mac and linux machines, with python 2.7, 3.6 and 3.7. Please try to install in a clean environment with appropriate dependencies.

[rainwoodman]

Does runtest.sh script use the source in tree or the currently installed version of software?

[rainwoodman]

It is because anaconda installed the MKL package. This issue was documented in the installation guide.

[rainwoodman]

My checklist is all cleared. All the best!

[mikaem]

@VivianePons I think that's a wrap!

[rainwoodman]

@mikaem What are the 'hard to address MKL bugs' mentioned in the doc?
If you can file it as an issue then maybe some day someone can look into it.

[VivianePons]

@whedon check references

[whedon]

Attempting to check references...

[whedon]

```Reference check summary:

OK DOIs

• 10.21105/joss.01071 is OK
• 10.1016/j.jpdc.2007.09.005 is OK
• 10.1109/MCSE.2010.118 is OK
• 10.1016/j.cpc.2016.02.005 is OK

MISSING DOIs

• https://doi.org/10.1016/j.jpdc.2019.02.006 may be missing for title: Fast parallel multidimensional FFT using advanced MPI
• https://doi.org/10.1109/jproc.2004.840301 may be missing for title: The Design and Implementation of FFTW3

INVALID DOIs

• None
  ```

[VivianePons]

@VivianePons I think that's a wrap!

Perfect! I will take care of the last editorial details very soon.

[mikaem]

@VivianePons I have updated a few of the references in the latest commit to master. Not sure about the missing titles above?

[mikaem]

@whedon generate pdf

[whedon]

Attempting PDF compilation. Reticulating splines etc...

[whedon]

:point_right: Check article proof :page_facing_up: :point_left:

[mikaem]

There is a doi for version 2.0.1 here: 10.5281/zenodo.2613971

[mikaem]

@mikaem What are the 'hard to address MKL bugs' mentioned in the doc?
If you can file it as an issue then maybe some day someone can look into it.

@rainwoodman See, e.g., this link. I have not looked further into it, but I agree, we should file an issue.

[VivianePons]

@whedon check references

[whedon]

Attempting to check references...

[whedon]

```Reference check summary:

OK DOIs

• 10.1016/j.jpdc.2019.02.006 is OK
• 10.21105/joss.01071 is OK
• 10.1016/j.jpdc.2007.09.005 is OK
• 10.1109/MCSE.2010.118 is OK
• 10.1016/j.cpc.2016.02.005 is OK

MISSING DOIs

• https://doi.org/10.1109/jproc.2004.840301 may be missing for title: The Design and Implementation of FFTW3

INVALID DOIs

• None
  ```

[kyleniemeyer]

Hi @mikaem, it looks like the Frigo and Johnson (2005) reference should include the DOI 10.1109/jproc.2004.840301. Could you add this?

[mikaem]

@whedon check references

[whedon]

Attempting to check references...

[whedon]

```Reference check summary:

OK DOIs

• 10.1016/j.jpdc.2019.02.006 is OK
• 10.21105/joss.01071 is OK
• 10.1016/j.jpdc.2007.09.005 is OK
• 10.1109/MCSE.2010.118 is OK
• 10.1109/JPROC.2004.840301 is OK
• 10.1016/j.cpc.2016.02.005 is OK

MISSING DOIs

• None

INVALID DOIs

• None
  ```

[mikaem]

@kyleniemeyer Done!

[mikaem]

@whedon generate pdf

[whedon]

Attempting PDF compilation. Reticulating splines etc...

[whedon]

:point_right: Check article proof :page_facing_up: :point_left:

[VivianePons]

Can you please:

• update the archive with the latest changes
• update the archive metainfo so that the title is the same than the paper title
• Add David Elliot Keyes to the list of authors in the archive

[mikaem]

@VivianePons Created a new release and there's a new doi: 10.5281/zenodo.2621442
This one has all three authors and the same title as the paper.

[mikaem]

@whedon set version 2.0.2

[mikaem]

@whedon set 2.0.2 as version

[whedon]

I'm sorry @mikaem, I'm afraid I can't do that. That's something only editors are allowed to do.

[mikaem]

@VivianePons Looks like you need to update the version to 2.0.2

[VivianePons]

@whedon set 2.0.2 as version

[whedon]

OK. 2.0.2 is the version.

[VivianePons]

@whedon set 10.5281/zenodo.2621442 as archive

[whedon]

OK. 10.5281/zenodo.2621442 is the archive.

[VivianePons]

@whedon generate pdf

[whedon]

Attempting PDF compilation. Reticulating splines etc...

[whedon]

:point_right: Check article proof :page_facing_up: :point_left:

[VivianePons]

Congratulations, this paper is good to go! Thank you @mikaem for this submission and @iljah and @rainwoodman for the reviews!

@openjournals/joss-eics it's up to you

[kyleniemeyer]

@whedon accept

[whedon]

Attempting dry run of processing paper acceptance...

[whedon]

Check final proof :point_right: https://github.com/openjournals/joss-papers/pull/598

If the paper PDF and Crossref deposit XML look good in https://github.com/openjournals/joss-papers/pull/598, then you can now move forward with accepting the submission by compiling again with the flag deposit=true e.g.
@whedon accept deposit=true

[whedon]

```Reference check summary:

OK DOIs

• 10.1016/j.jpdc.2019.02.006 is OK
• 10.21105/joss.01071 is OK
• 10.1016/j.jpdc.2007.09.005 is OK
• 10.1109/MCSE.2010.118 is OK
• 10.1109/JPROC.2004.840301 is OK
• 10.1016/j.cpc.2016.02.005 is OK

MISSING DOIs

• None

INVALID DOIs

• None
  ```

[kyleniemeyer]

@whedon accept deposit=true

[whedon]

Doing it live! Attempting automated processing of paper acceptance...

[whedon]

🚨🚨🚨 THIS IS NOT A DRILL, YOU HAVE JUST ACCEPTED A PAPER INTO JOSS! 🚨🚨🚨

Here's what you must now do:

1. Check final PDF and Crossref metadata that was deposited :point_right: https://github.com/openjournals/joss-papers/pull/599
2. Wait a couple of minutes to verify that the paper DOI resolves https://doi.org/10.21105/joss.01340
3. If everything looks good, then close this review issue.

4. Party like you just published a paper! 🎉🌈🦄💃👻🤘

   Any issues? notify your editorial technical team...

[kyleniemeyer]

@mikaem congrats on your JOSS publication, and thanks to @iljah and @rainwoodman for reviewing and @VivianePons for editing!

[whedon]

:tada::tada::tada: Congratulations on your paper acceptance! :tada::tada::tada:

If you would like to include a link to your paper from your README use the following code snippets:

Markdown:
[![DOI](http://joss.theoj.org/papers/10.21105/joss.01340/status.svg)](https://doi.org/10.21105/joss.01340)

HTML:
<a style="border-width:0" href="https://doi.org/10.21105/joss.01340">
  <img src="http://joss.theoj.org/papers/10.21105/joss.01340/status.svg" alt="DOI badge" >
</a>

reStructuredText:
.. image:: http://joss.theoj.org/papers/10.21105/joss.01340/status.svg
   :target: https://doi.org/10.21105/joss.01340

This is how it will look in your documentation:

We need your help!

Journal of Open Source Software is a community-run journal and relies upon volunteer effort. If you'd like to support us please consider doing either one (or both) of the the following:

• Volunteering to review for us sometime in the future. You can add your name to the reviewer list here: http://joss.theoj.org/reviewer-signup.html
• Making a small donation to support our running costs here: https://numfocus.salsalabs.org/donate-to-joss