Joss-reviews: [REVIEW]: pyrgg: Python Random Graph Generator

[whedon]

Submitting author: @sepandhaghighi (Sepand Haghighi)
Repository: https://github.com/sepandhaghighi/pyrgg
Version: v0.1
Editor: @danielskatz
Reviewer: @hausen
Archive: 10.5281/zenodo.845502

Status

Status badge code:

HTML: <a href="http://joss.theoj.org/papers/da33f691984d9a35f66ff93a391bbc26"><img src="http://joss.theoj.org/papers/da33f691984d9a35f66ff93a391bbc26/status.svg"></a>
Markdown: [![status](http://joss.theoj.org/papers/da33f691984d9a35f66ff93a391bbc26/status.svg)](http://joss.theoj.org/papers/da33f691984d9a35f66ff93a391bbc26)

Reviewers and authors:

Please avoid lengthy details of difficulties in the review thread. Instead, please create a new issue in the target repository and link to those issues (especially acceptance-blockers) in the review thread below. (For completists: if the target issue tracker is also on GitHub, linking the review thread in the issue or vice versa will create corresponding breadcrumb trails in the link target.)

Reviewer questions

@hausen, please carry out your review in this issue by updating the checklist below (please make sure you're logged in to GitHub). The reviewer guidelines are available here: http://joss.theoj.org/about#reviewer_guidelines. Any questions/concerns please let @danielskatz know.

Conflict of interest

• [x] As the reviewer I confirm that there are no conflicts of interest for me to review this work (such as being a major contributor to the software).

Code of Conduct

• [x] I confirm that I read and will adhere to the JOSS code of conduct.

General checks

• [x] Repository: Is the source code for this software available at the repository url?
• [x] License: Does the repository contain a plain-text LICENSE file with the contents of an OSI approved software license?
• [x] Version: Does the release version given match the GitHub release (v0.1)?
• [x] Authorship: Has the submitting author (@sepandhaghighi) made major contributions to the software?

Functionality

• [x] Installation: Does installation proceed as outlined in the documentation?
• [x] Functionality: Have the functional claims of the software been confirmed?
• [x] Performance: Have any performance claims of the software been confirmed?

Documentation

• [ ] A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
• [x] Installation instructions: Is there a clearly-stated list of dependencies? Ideally these should be handled with an automated package management solution.
• [ ] Example usage: Do the authors include examples of how to use the software (ideally to solve real-world analysis problems).
• [x] Functionality documentation: Is the core functionality of the software documented to a satisfactory level (e.g., API method documentation)?
• [x] Automated tests: Are there automated tests or manual steps described so that the function of the software can be verified?
• [x] Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support

Software paper

• [x] Authors: Does the paper.md file include a list of authors with their affiliations?
• [ ] A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
• [x] References: Do all archival references that should have a DOI list one (e.g., papers, datasets, software)?

[whedon]

Hello human, I'm @whedon. I'm here to help you with some common editorial tasks for JOSS. @hausen it looks like you're currently assigned as the reviewer for this paper :tada:.

:star: Important :star:

If you haven't already, you should seriously consider unsubscribing from GitHub notifications for this (https://github.com/openjournals/joss-reviews) repository. As as reviewer, you're probably currently watching this repository which means for GitHub's default behaviour you will receive notifications (emails) for all JOSS reviews 😿

To fix this do the following two things:

1. Set yourself as 'Not watching' https://github.com/openjournals/joss-reviews:

1. You may also like to change your default settings for this watching repositories in your GitHub profile here: https://github.com/settings/notifications

For a list of things I can do to help you, just type:

@whedon commands

[danielskatz]

@hausen - this is ready for you to review.

Please carry out your review in this issue by updating the checklist above (please make sure you're logged in to GitHub)

The reviewer guidelines are available here: http://joss.theoj.org/about#reviewer_guidelines

Please avoid lengthy details of difficulties in this review thread. Instead, please create a new issue in the target repository and link to those issues (especially acceptance-blockers) in this review thread. (For completists: if the target issue tracker is also on GitHub, linking the review thread in the issue or vice versa will create corresponding breadcrumb trails in the link target.)

Any questions/concerns, please let me know.

Thanks!!

[hausen]

Issues: https://github.com/sepandhaghighi/pyrgg/issues/2

[danielskatz]

@hausen - looking at the comments in https://github.com/sepandhaghighi/pyrgg/issues/2, it seems like is ready for your re-review

[danielskatz]

@hausen - have you had a chance to take a look at this again?

[hausen]

@danielskatz , was busy for a couple of days, sorry. I've just re-reviewed it. There are still some issues.

[hausen]

The author states that the software was created to fill a specific need of his lab and that it is being used to generate samples for benchmarking some undisclosed algorithms, the results of which will be released _in the future_.

Therefore, "statement of need," "example usage" and "references" are still missing.

Moreover, IMHO, the software is too simple to be of general use as it is.

If the author disagrees, he should disclose more details of what the software is being used for, maybe by putting a preprint of the benchmarking results on Arxiv and showing how they were obtained.

[sepandhaghighi]

Hi

I tell a fact, I have a research project that need graphs ( different sizes )
in DIMACS/JSON format, and my framework written in python, what can I do ? there is no python package that generate graphs in this format. I think this fact is enough for statement of need.(Is not specific need, most of graph processing frameworks in hardware level use DIMACS/JSON format)

I'm not owner of research projects mentioned before, so unfortunately is not possible to share and publish any parts of them.

Your opinion is respected but I think you take a little hard, you tell that pyrgg is too simple, I don't know whats the problem ? There are lots of software and packages that are not complex but there are useful in research projects.

most of converters and generators in research projects focused on formatting and automation so there are not complex in algorithm, like country converter coco that accepted in joss, if we want to summarized its like a simple table but it is very useful.

I think at least there is no problem with my reference, both of them have DOI .

Best Regards
Sepand Haghighi

[danielskatz]

@sepandhaghighi - please discuss in general, in your documentation, what you expect others to do with the software, and how this will support their research. They can be hypothetical.

Note this is intended to answer: "Do the authors clearly state what problems the software is designed to solve and who the target audience is?"

And you do need to provide some examples of usage.

[danielskatz]

@hausen - Regarding references, please be aware this is a question about the references in the paper itself: Are there appropriate references (citations), and do they have DOIs if there are DOIs for them?

[danielskatz]

@hausen - For the references item in the checklist:

Are there appropriate references (citations), and do they have DOIs if there are DOIs for them?

If so, please check off the references item in the checklist. If not, please let us know what you think is needed.

[danielskatz]

@sepandhaghighi - please note some inputs from you are needed. As stated above:

Please discuss in general, in your documentation, what you expect others to do with the software, and how this will support their research. They can be hypothetical.

Note this is intended to answer: "Do the authors clearly state what problems the software is designed to solve and who the target audience is?"

And you do need to provide some examples of usage.

[hausen]

@danielskatz , it seems that the author is pushing some changes to the repository. I'll wait until he is done and check the references again.

[sepandhaghighi]

Hi

Sorry for my late reply
I added more features to project (new file formats, update documents and webpage, code coverage and ...)

1-Statement of need : I added this section to Readme and Paper :

pyrgg is an easy to use synthetic random graph generator written in python, that support variant graph file formats include DIMACS(.gr) pyrgg has the ability to generate graphs in different sizes and designed to provide input files for test, benchmark and performance-analysis of graph processing frameworks, but it can be used in any other research program that include graphs with these formats.

2- Example Of Usage : I added Example Of Usage section to Readme

3- References are available in paper.bib

Best Regards
Sepand Haghighi

[danielskatz]

@hausen - back to you

[hausen]

@sepandhaghighi

1 - Statement of need: I can understand the text, but it would be better if you had someone proofread it.

2 - Example of usage: Can you please provide detailed instructions on how you produced the data for the bar graph in the "example of usage" section?

3 - References: ok

[sepandhaghighi]

@hausen

1 - Statement of need : I've modified the text -->

Pyrgg is an easy-to-use synthetic random graph generator written in Python which supports various graph file formats including DIMACS .gr files.
Pyrgg has the ability to generate graphs of different sizes and is designed to provide input files for broad range of graph-based research applications, including but not limited to testing, benchmarking and performance-analysis of graph processing frameworks.

2 - Example of usage : I added Table. 1. Characteristics of graphs & Fig. 2. Rand Graph Generation in Example of Usage section .
Fig. 1(bar graph) is about runtime comparison between Medusa and pure CUDA implementation of SSSP for different graphs.

Best Regards
Sepand Haghighi

[hausen]

@sepandhaghighi, the instructions only state how to generate the random graphs. How is the data for the bar graphs obtained?

[danielskatz]

@sepandhaghighi:

For statement of need, both the paper and the software documentation should state "what problems the software is designed to solve and who the target audience is?" Think about this from the point-of-view of someone who is not an expert in your software. How will they know if they should use this software?

For example of usage, the software documentation needs to "include examples of how to use the software (ideally to solve real-world analysis problems)." Think again of a person who is not familiar with you software, and provide step-by-step examples of how they would use your software. Someone, such as a reviewer, should be able to follow these steps to run the software and obtain results similar to those you have obtained.

[sepandhaghighi]

@hausen , Bar graph data obtained from Medusa and pure CUDA implementation of SSSP for different graphs, and these steps are out of this paper scope(related to other frameworks). I added Fig.1 only to show this benchmark includes random graph.

[sepandhaghighi]

@danielskatz :

1. statement of need

pyrgg is an easy-to-use synthetic random graph generator written in Python which supports various graph file formats including DIMACS .gr files. pyrgg has the ability to generate graphs of different sizes and is designed to provide input files for broad range of graph-based research applications, including but not limited to testing, benchmarking and performance-analysis of graph processing frameworks.

I think it is clear, isn't ?

1. Example of usage
   I added step-by-step examples of how generate graphs with different formats and also described each format separately but next steps (about graph analysis) are out of this paper scope (related to other frameworks).

[danielskatz]

Can you add to the paper and the documentation something that helps say "who the target audience is"? What types of researchers might use this software? Is this computer scientists who study graph algorithms? Data scientists who build machine learning frameworks? Users in a particular field, such as bioinformatics?

[danielskatz]

Personally, I think the examples of usage part is fine, with the videos. It's not exactly "(ideally to solve real-world analysis problems)" but this software doesn't solve analysis problems, so it's ok.

[sepandhaghighi]

@danielskatz I updated the text :

Pyrgg is an easy-to-use synthetic random graph generator written in Python which supports various graph file formats including DIMACS .gr files. Pyrgg has the ability to generate graphs of different sizes and is designed to provide input files for broad range of graph-based research applications, including but not limited to testing, benchmarking and performance-analysis of graph processing frameworks. Pyrgg target audiences are computer scientists who study graph algorithms and graph processing frameworks.

[danielskatz]

Thanks @sepandhaghighi

@hausen - I think this is sufficient for the statements of need, and that the example usage is also ok, but I defer to you to make your recommendation.

[hausen]

Well, that's probably as good as it gets without a real-world example of usage.

My last two comments:

1) since it's "out of this paper scope" to show how the data for Fig 1 and Table 1 was processed, please remove them both from the README.md. IMHO, displaying this information in the README -- without at least mentioning how you went from the output of the program to the data that gave birth to it -- only confuses a potential user of your program.

2) in the future, do update the repository with links to published research results that used your code

Once 1 is done, we're good to go!

[sepandhaghighi]

@hausen

1. Removed

2. I will add links in the future.

[hausen]

Gentlemen, I think it's a wrap. Back to you, @danielskatz.

[danielskatz]

Thanks @hausen for your reviewing work, and @sepandhaghighi for your paper, software and updates.

@arfon - over to you.

[arfon]

@sepandhaghighi - the citations aren't being compiled in by Pandoc as you're not citing them in the body of the paper. Could you cite them directly please? (You can read how to do that here)

[sepandhaghighi]

@arfon Done ;-)

[arfon]

@sepandhaghighi - At this point could you make an archive of the reviewed software in Zenodo/figshare/other service and update this thread with the DOI of the archive? I can then move forward with accepting the submission.

[sepandhaghighi]

@arfon

10.5281/zenodo.845502

[arfon]

@whedon set 10.5281/zenodo.845502 as archive

[arfon]

@whedon set 10.5281/zenodo.845502 as archive

[whedon]

OK. 10.5281/zenodo.845502 is the archive.

[arfon]

@hausen - many thanks for your review here and to @danielskatz for editing this submission ✨

@sepandhaghighi - your paper is now accepted into JOSS and your DOI is http://dx.doi.org/10.21105/joss.00331 ⚡️ 🚀 💥

[sepandhaghighi]

@hausen @danielskatz @arfon Thanks ;-)