Joss-reviews: [REVIEW]: DEEPaaS API: a REST API for Machine Learning and Deep Learning models

[whedon]

Submitting author: @alvarolopez (Álvaro López García)
Repository: https://github.com/indigo-dc/DEEPaaS
Version: 0.5.2
Editor: @trallard
Reviewer: @krother, @rougier
Archive: 10.5281/zenodo.3519350

Status

Status badge code:

HTML: <a href="http://joss.theoj.org/papers/b6c4f836a5c8573219ceb8ca23f7b00c"><img src="http://joss.theoj.org/papers/b6c4f836a5c8573219ceb8ca23f7b00c/status.svg"></a>
Markdown: [![status](http://joss.theoj.org/papers/b6c4f836a5c8573219ceb8ca23f7b00c/status.svg)](http://joss.theoj.org/papers/b6c4f836a5c8573219ceb8ca23f7b00c)

Reviewers and authors:

Please avoid lengthy details of difficulties in the review thread. Instead, please create a new issue in the target repository and link to those issues (especially acceptance-blockers) by leaving comments in the review thread below. (For completists: if the target issue tracker is also on GitHub, linking the review thread in the issue or vice versa will create corresponding breadcrumb trails in the link target.)

Reviewer instructions & questions

@dangunter & @krother & @rougier, please carry out your review in this issue by updating the checklist below. If you cannot edit the checklist please:

1. Make sure you're logged in to your GitHub account
2. Be sure to accept the invite at this URL: https://github.com/openjournals/joss-reviews/invitations

The reviewer guidelines are available here: https://joss.readthedocs.io/en/latest/reviewer_guidelines.html. Any questions/concerns please let @trallard know.

✨ Please try and complete your review in the next two weeks ✨

Review checklist for @dangunter

Conflict of interest

• [ ] As the reviewer I confirm that I have read the JOSS conflict of interest policy and that there are no conflicts of interest for me to review this work.

Code of Conduct

• [ ] I confirm that I read and will adhere to the JOSS code of conduct.

General checks

• [ ] Repository: Is the source code for this software available at the repository url?
• [ ] License: Does the repository contain a plain-text LICENSE file with the contents of an OSI approved software license?
• [ ] Version: 0.5.2
• [ ] Authorship: Has the submitting author (@alvarolopez) made major contributions to the software? Does the full list of paper authors seem appropriate and complete?

Functionality

• [ ] Installation: Does installation proceed as outlined in the documentation?
• [ ] Functionality: Have the functional claims of the software been confirmed?
• [ ] Performance: If there are any performance claims of the software, have they been confirmed? (If there are no claims, please check off this item.)

Documentation

• [ ] A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
• [ ] Installation instructions: Is there a clearly-stated list of dependencies? Ideally these should be handled with an automated package management solution.
• [ ] Example usage: Do the authors include examples of how to use the software (ideally to solve real-world analysis problems).
• [ ] Functionality documentation: Is the core functionality of the software documented to a satisfactory level (e.g., API method documentation)?
• [ ] Automated tests: Are there automated tests or manual steps described so that the function of the software can be verified?
• [ ] Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support

Software paper

• [ ] Authors: Does the paper.md file include a list of authors with their affiliations?
• [ ] A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
• [ ] References: Do all archival references that should have a DOI list one (e.g., papers, datasets, software)?

Review checklist for @krother

Conflict of interest

• [x] As the reviewer I confirm that I have read the JOSS conflict of interest policy and that there are no conflicts of interest for me to review this work.

Code of Conduct

• [x] I confirm that I read and will adhere to the JOSS code of conduct.

General checks

• [x] Repository: Is the source code for this software available at the repository url?
• [x] License: Does the repository contain a plain-text LICENSE file with the contents of an OSI approved software license?
• [x] Version: 0.5.2
• [x] Authorship: Has the submitting author (@alvarolopez) made major contributions to the software? Does the full list of paper authors seem appropriate and complete?

Functionality

• [x] Installation: Does installation proceed as outlined in the documentation?
• [x] Functionality: Have the functional claims of the software been confirmed?
• [x] Performance: If there are any performance claims of the software, have they been confirmed? (If there are no claims, please check off this item.)

Documentation

• [x] A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
• [x] Installation instructions: Is there a clearly-stated list of dependencies? Ideally these should be handled with an automated package management solution.
• [x] Example usage: Do the authors include examples of how to use the software (ideally to solve real-world analysis problems).
• [x] Functionality documentation: Is the core functionality of the software documented to a satisfactory level (e.g., API method documentation)?
• [x] Automated tests: Are there automated tests or manual steps described so that the function of the software can be verified?
• [x] Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support

Software paper

• [x] Authors: Does the paper.md file include a list of authors with their affiliations?
• [x] A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
• [x] References: Do all archival references that should have a DOI list one (e.g., papers, datasets, software)?

Review checklist for @rougier

Conflict of interest

• [x] As the reviewer I confirm that I have read the JOSS conflict of interest policy and that there are no conflicts of interest for me to review this work.

Code of Conduct

• [x] I confirm that I read and will adhere to the JOSS code of conduct.

General checks

• [x] Repository: Is the source code for this software available at the repository url?
• [x] License: Does the repository contain a plain-text LICENSE file with the contents of an OSI approved software license?
• [x] Version: 0.5.2
• [x] Authorship: Has the submitting author (@alvarolopez) made major contributions to the software? Does the full list of paper authors seem appropriate and complete?

Functionality

• [x] Installation: Does installation proceed as outlined in the documentation?
• [x] Functionality: Have the functional claims of the software been confirmed?
• [ ] Performance: If there are any performance claims of the software, have they been confirmed? (If there are no claims, please check off this item.)

Documentation

• [x] A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
• [x] Installation instructions: Is there a clearly-stated list of dependencies? Ideally these should be handled with an automated package management solution.
• [x] Example usage: Do the authors include examples of how to use the software (ideally to solve real-world analysis problems).
• [x] Functionality documentation: Is the core functionality of the software documented to a satisfactory level (e.g., API method documentation)?
• [x] Automated tests: Are there automated tests or manual steps described so that the function of the software can be verified?
• [x] Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support

Software paper

• [x] Authors: Does the paper.md file include a list of authors with their affiliations?
• [x] A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
• [x] References: Do all archival references that should have a DOI list one (e.g., papers, datasets, software)?

[whedon]

Hello human, I'm @whedon, a robot that can help you with some common editorial tasks. @dangunter, @krother, @rougier it looks like you're currently assigned to review this paper :tada:.

:star: Important :star:

If you haven't already, you should seriously consider unsubscribing from GitHub notifications for this (https://github.com/openjournals/joss-reviews) repository. As a reviewer, you're probably currently watching this repository which means for GitHub's default behaviour you will receive notifications (emails) for all reviews 😿

To fix this do the following two things:

1. Set yourself as 'Not watching' https://github.com/openjournals/joss-reviews:

1. You may also like to change your default settings for this watching repositories in your GitHub profile here: https://github.com/settings/notifications

For a list of things I can do to help you, just type:

@whedon commands

[whedon]

Attempting PDF compilation. Reticulating splines etc...

[whedon]

:point_right: Check article proof :page_facing_up: :point_left:

[trallard]

Hey @dangunter, @krother, @rougier thanks a lot for offering to review this submission.

Each of you have been assigned a checklist for your review, please go over the items and check them if you consider them to be satisfied.
Please use this issue for ongoing discussions. If you need to start a longer discussion or request please make an issue on the submission repo and reference it here.

If you encounter any issues or have questions, please feel free to ping me here.

Thanks again for your help 🙏🏼

[trallard]

@whedon remind @rougier in 2 weeks

[whedon]

Reminder set for @rougier in 2 weeks

[trallard]

Hi @krother @dangunter I am just popping by to see how the review is going and check if you need additional help

[whedon]

:wave: @rougier, please update us on how your review is going.

[rougier]

Thanks for the reminder. I will try to finish by the end of the week.

[rougier]

@trallard Can we have a link to the paper at the top of the thread ? (I cannot find it)

[trallard]

@whedon generate pdf

[whedon]

Attempting PDF compilation. Reticulating splines etc...

[whedon]

:point_right: Check article proof :page_facing_up: :point_left:

[trallard]

@rougier here is the article 👆🏼

[rougier]

@alvarolopez I'm trying to test your software but I have problems because of the gevent package that cannot compile v1.4.0 on my machine (OSX). Last version seems to compile ok but not the 1.4.0. Do you know through which package this dependency comes from ?

[alvarolopez]

@rougier this is a dependency on DEEPaaS itself, but we are depending on version 1.2.0 (actually I now realise that this can be bumped to 1.3.0).

What error are you obtaining?

[rougier]

Lots of warnings and errors which make it difficult to track the actual error. Among them:

  src/gevent/libev/gevent.corecext.c:30182:21: error: no member named 'exc_type' in 'struct _ts'
      *type = tstate->exc_type;
              ~~~~~~  ^
  src/gevent/libev/gevent.corecext.c:30183:22: error: no member named 'exc_value' in 'struct _ts'; did you mean 'curexc_value'?
      *value = tstate->exc_value;
                       ^~~~~~~~~

[alvarolopez]

@rouger I have fixed this issue, related with an old pinned dependency version and Python3. Version 0.5.0 should work.

[rougier]

I confirm it fixed my problem.

[rougier]

@alvarolopez I think you're missing some DOI in the references of your paper. Also, I did not find any way to test the package. Would you that be possible to have a step by step description of a minimal test to run?

[trallard]

Thanks all for the reviews.
@dangunter, @krother can I have an estimated date for this review to be completed? it has been going on for a while and we'd like to move it forwards

[alvarolopez]

Hi @rougier, catching up after summer break.

I will correct the DOIs as soon as possible.

Regarding the instructions to run the API is to use the "make develop" command, documented here I reckon that docs must be updated properly, with more detailed information. Currently there is more information on how to run it in the project-wide documentation here.

If you want to test a real model that is integrated with the API, you can have a loot at here.

[trallard]

Hi folks, just checking in to see how is the review moving? @alvarolopez

[alvarolopez]

@trallard I've fixed the missing DOI in several publications, under the paper branch.

[rougier]

Catching up. Would it be possible to have a (even minimal) set of unittests to check everything has been installed properly and is running as expected ?

[alvarolopez]

@rougier unit tests do exist (https://github.com/indigo-dc/DEEPaaS/tree/master/deepaas/tests) with a coverage of about 80%. A full fledged testing is being done using Jenkins: https://jenkins.indigo-datacloud.eu/job/Pipeline-as-code/job/DEEPaaS/job/master/

[alvarolopez]

@whedon generate pdf

[whedon]

Attempting PDF compilation. Reticulating splines etc...

[whedon]

:point_right: Check article proof :page_facing_up: :point_left:

[alvarolopez]

@whedon generate pdf

[whedon]

Attempting PDF compilation. Reticulating splines etc...

[whedon]

:point_right: Check article proof :page_facing_up: :point_left:

[alvarolopez]

@whedon generate pdf from branch paper

[whedon]

Attempting PDF compilation from custom branch paper. Reticulating splines etc...

[whedon]

:point_right: Check article proof :page_facing_up: :point_left:

[rougier]

@alvarolopez Yep. Sorry, I did not see them.

[rougier]

@trallard @alvarolopez With that last point (unit tests) and the DOI correction I think the paper is ready to be published on my side. Any news from @dangunter and @krother?

[alvarolopez]

@whedon generate pdf from branch joss-paper

[whedon]

Attempting PDF compilation from custom branch joss-paper. Reticulating splines etc...

[whedon]

:point_right: Check article proof :page_facing_up: :point_left:

[alvarolopez]

@rougier I finally managed to get the correct DOIs being rendered, you can check the proofs here: https://github.com/openjournals/joss-papers/blob/joss.01517/joss.01517/10.21105.joss.01517.pdf

[krother]

Looks good, ready to go.

[alvarolopez]

@krother, @rougier many thanks.

@trallard , what are the next steps to be done?

[trallard]

It seems like @dangunter has gone MIA but since we have the reviews from @krother and @rougier if they are happy for me to proceed / recommend acceptance I am happy to move this forward

---

In the meantime @alvarolopez can you perform the following actions:

1. Create a new release if needed and update us on the final version of the software
2. Check again the paper for typos, missing DOIs and the such
3. Create an archive (on Zenodo, figshare, or other) and post the archive DOI on this issue.

Once this is completed we can move over 😄

[trallard]

@whedon generate pdf

[whedon]

Attempting PDF compilation. Reticulating splines etc...

[whedon]

:point_right: Check article proof :page_facing_up: :point_left:

[alvarolopez]

@trallard ok, what I would do is:

1. Merge the branch paper into the master branch so that the paper is built correctly.
2. Prepare a new release
3. Upload the new version into https://digital.csic.es/handle/10261/173108, that is our institutional repository. I need to check if a new version can be added there or if I need a new entry.

I will keep you posted on this.

[alvarolopez]

@whedon generate pdf

[whedon]

Attempting PDF compilation. Reticulating splines etc...

[whedon]

:point_right: Check article proof :page_facing_up: :point_left:

[alvarolopez]

@whedon generate pdf

[whedon]

Attempting PDF compilation. Reticulating splines etc...

[whedon]

:point_right: Check article proof :page_facing_up: :point_left:

[alvarolopez]

@trallard ok, what I would do is:

1. Merge the branch `paper` into the `master` branch so that the paper is built correctly.

2. Prepare a new release

3. Upload the new version into https://digital.csic.es/handle/10261/173108, that is our institutional repository. I need to check if a new version can be added there or if I need a new entry.

I will keep you posted on this.

@trallard I think we are ready to go. The paper in the master branch is updated, and the Zenodo record is here: https://doi.org/10.5281/zenodo.1544377

Anything else from our side?

[alvarolopez]

@trallard hi again. is there anything else we can do to speed up this (final) process?

[trallard]

@whedon check references

[whedon]

Attempting to check references...

[whedon]

```Reference check summary:

OK DOIs

• 10.1007/s10462-018-09679-z is OK
• 10.1126/science.1110411 is OK

MISSING DOIs

• None

INVALID DOIs

• https://doi.org/10.1016/j.dss.2012.05.045 is INVALID because of 'https://doi.org/' prefix
  ```

[trallard]

@whedon set 0.5.2 as version

[whedon]

OK. 0.5.2 is the version.

[trallard]

@alvarolopez thanks for your patience.
I have made 2 PRs to add some changes prior to publication, one is to fix the broken reference as per above and the other are minimal suggestions (can or not approve this one).

Please ping me when the DOI fix is merged to proceed.

[alvarolopez]

@trallard ping :-)

Changes were merged, thank you for your support!

[trallard]

@whedon check references

[whedon]

Attempting to check references...

[whedon]

```Reference check summary:

OK DOIs

• 10.1007/s10462-018-09679-z is OK
• 10.1126/science.1110411 is OK
• 10.1016/j.dss.2012.05.045 is OK

MISSING DOIs

• None

INVALID DOIs

• None
  ```

[trallard]

Ok I think this review is ready to be sent over to @openjournals/joss-eics for their editorial duties.
I am recommending acceptance now that the reviews and my editorial checks have been completed.

[kthyng]

thanks @trallard!

@alvarolopez Please edit the metadata in your Zenodo archive so that the title and authors exactly match your paper here.

[alvarolopez]

@kthyng done, check https://doi.org/10.5281/zenodo.1544377

[kthyng]

@alvarolopez Actually can you check again? The title in Zenodo should be "DEEPaaS API: a REST API for Machine Learning and Deep Learning models" right?

[alvarolopez]

@kthyng yes indeed, my mistake. Now it is adjusted.

[kthyng]

@alvarolopez Ok great, though the version is now inconsistent between this and Zenodo. Here, it is listed as 0.5.2, which is the latest release on github. Is that correct? If so, please update it on Zenodo to match.

[alvarolopez]

@kthyng :man_facepalming: yes indeed. I forgot to create the release for that version.

[kthyng]

@whedon accept

[whedon]

No archive DOI set. Exiting...

[kthyng]

Oh right

[kthyng]

@whedon set 10.5281/zenodo.3519350 as archive

[whedon]

OK. 10.5281/zenodo.3519350 is the archive.

[kthyng]

@whedon accept

[whedon]

Attempting dry run of processing paper acceptance...

[whedon]

```Reference check summary:

OK DOIs

• 10.1007/s10462-018-09679-z is OK
• 10.1126/science.1110411 is OK
• 10.1016/j.dss.2012.05.045 is OK

MISSING DOIs

• None

INVALID DOIs

• None
  ```

[whedon]

Check final proof :point_right: https://github.com/openjournals/joss-papers/pull/1055

If the paper PDF and Crossref deposit XML look good in https://github.com/openjournals/joss-papers/pull/1055, then you can now move forward with accepting the submission by compiling again with the flag deposit=true e.g.
@whedon accept deposit=true

[kthyng]

@whedon accept deposit=true

[whedon]

Doing it live! Attempting automated processing of paper acceptance...

[whedon]

🐦🐦🐦 👉 Tweet for this paper 👈 🐦🐦🐦

[whedon]

🚨🚨🚨 THIS IS NOT A DRILL, YOU HAVE JUST ACCEPTED A PAPER INTO JOSS! 🚨🚨🚨

Here's what you must now do:

1. Check final PDF and Crossref metadata that was deposited :point_right: https://github.com/openjournals/joss-papers/pull/1056
2. Wait a couple of minutes to verify that the paper DOI resolves https://doi.org/10.21105/joss.01517
3. If everything looks good, then close this review issue.

4. Party like you just published a paper! 🎉🌈🦄💃👻🤘

   Any issues? notify your editorial technical team...

[kthyng]

Congrats @alvarolopez on your new paper!!

Many thanks to @krother and @rougier for your time and expertise reviewing, and to @trallard for editing!

[kthyng]

@arfon I just realized that the final paper still lists as a reviewer someone who ended up not participated but is still listed. Can we remove them?

[arfon]

@arfon I just realized that the final paper still lists as a reviewer someone who ended up not participated but is still listed. Can we remove them?

Sure thing, I can fix that.

[arfon]

OK done!

[alvarolopez]

@trallard @kthyng @krother @rougier thanks for taking care of the whole process.

@arfon: I just realized that my surname and name are not correctly identified. My name is "Álvaro" and surname is "López García" (i.e. I have two surnames). Can this be fixed?

Many thanks in advance.

[arfon]

@arfon: I just realized that my surname and name are not correctly identified. My name is "Álvaro" and surname is "López García" (i.e. I have two surnames). Can this be fixed?

Sorry about that @alvarolopez. That should be fixed now. If the PDF on the JOSS site is still looking out of date it might take a while to refresh as there's caching in place.

[whedon]

:tada::tada::tada: Congratulations on your paper acceptance! :tada::tada::tada:

If you would like to include a link to your paper from your README use the following code snippets:

Markdown:
[![DOI](https://joss.theoj.org/papers/10.21105/joss.01517/status.svg)](https://doi.org/10.21105/joss.01517)

HTML:
<a style="border-width:0" href="https://doi.org/10.21105/joss.01517">
  <img src="https://joss.theoj.org/papers/10.21105/joss.01517/status.svg" alt="DOI badge" >
</a>

reStructuredText:
.. image:: https://joss.theoj.org/papers/10.21105/joss.01517/status.svg
   :target: https://doi.org/10.21105/joss.01517

This is how it will look in your documentation:

We need your help!

Journal of Open Source Software is a community-run journal and relies upon volunteer effort. If you'd like to support us please consider doing either one (or both) of the the following:

• Volunteering to review for us sometime in the future. You can add your name to the reviewer list here: https://joss.theoj.org/reviewer-signup.html
• Making a small donation to support our running costs here: https://numfocus.org/donate-to-joss

[kthyng]

Thanks @arfon! Sorry for all the after-changes.

[alvarolopez]

@arfon, many thanks!