Chapel: Document the Matrix Market package

[buddha314]

Right now there are not many ways to load data into Chapel. The MatrixMarket package is one of the few I've found besides just reading a file.

This package has no presence in the Chapel docs. And not knowing that both mmread() and mmreadsp() exist can cause a brother quite a bit of anxiety.

If MatrixMarket is not the preferred way to load sparse matrices, then please document the preferred way instead.

[buddha314]

Nota Bene: Perhaps mmread() should call mmreadsp() if the header indicates a sparse matrix.

Looks like the term "coordinate" indicates a sparse matrix according to the format specs

[bradcray]

Attn @ct-clmsn : Would you have time to add chpldoc documentation to the MatrixMarket package? Using the wrong routine (due to lack of documentation) caused @buddha314 to get OOM killers taking his program down, and for me it took all of my Mac apps down including work-in-progress terminals. I'm also curious about the question he asks regarding whether the mmread*() routines could be smarter about distinguishing between dense and sparse file formats. Specifically, his code pointed mmread() at a sparse matrix file and it tried to allocate the dense matrix, causing all sorts of misery.

A related question for the mason / documentation folks (so @ben-albrecht , @lydia-duncan , and maybe @benharsh and @psahabu ): The request for chpldocumentation here makes me wonder whether it would be possible for us to document any Mason packages that are listed in the centralized registry at chapel.cray.com so that it's searchable, unified, etc. I don't know if there's a precedent for this in the Rust/Cargo world, but I can imagine that @buddha314's "lack of documentation" problem will only get worse in a distributed package world.

[lydia-duncan]

I think @Spartee wanted some sort of documentation center for the Mason packages, but it sounded like the intention for that was to have it live in a separate place, and that it was lower priority than the other fixes

[lydia-duncan]

We could probably add a routine to our generation of the module documentation to pull down all the packages from Mason and put them in a sub-directory labeled as such (though that might not be sustainable as the number of packages grows)

[bradcray]

We could probably add a routine to our generation of the module documentation
to pull down all the packages from Mason and put them in a sub-directory
labeled as such

That's exactly what I was imagining. Bonus points if the documentation made it clear what the source of the package was in some way.

(though that might not be sustainable as the number of
packages grows

That sounds like a great problem to have.

[ben-albrecht]

As a reference, I believe Cargo defaults to generating documentation and hosting it here: https://docs.rs/

... with the option to provide your own documentation URL instead. Something like this would be my preference in the longer term, but if we want the docs added soon, we could build them into the module documentation.

Note that this would tie snapshots of the existing packages into the release documentation, and we would only be showing the latest package version's documentation on chapel.cray.com/docs/master

[ben-albrecht]

This could also be made into an issue on https://github.com/ct-clmsn/MatrixMarket - but we should keep an issue open on chapel for something like "Expose documentation for Mason packages", which tackles the questions brought up in this discussion.

[ben-albrecht]

This could also be made into an issue on https://github.com/ct-clmsn/MatrixMarket - but we should keep an issue open on chapel for something like "Expose documentation for Mason packages"

I'm going to close this issue, as the broader issue of "Expose documentation for Mason packages" exists now: https://github.com/chapel-lang/chapel/issues/10618