Stdlib: Documentation

[milancurcic]

Use this issue how to best document the stdlib. I put a placeholder Markdown file here.

[jvdp1]

Regarding the documention, is it planned to use some tools, like Doxygen or Ford to generate documention from the code?

[certik]

I think we should write documentation as comments in the code and generate nice html documentation from it, that way the documentation and code is at one place and easier to maintain and keep in sync. So whatever tool allows us to do that, or if we have to write or improve some tools, I think we should do that.

[zbeekman]

FORD is my preference unless someone knows a good way to do that with sphinx.

I think it can be worthwhile to have additional docs. I like hosting markdown based docs on RTD and API docs on gh-pages.

[certik]

Down the road I would like to use LFortran to execute "doctests" just like in Python. Here is the issue for it: https://gitlab.com/lfortran/lfortran/issues/73

[ivan-pi]

FORD also supports the option to include additional markdown based docs (http://d3denergetic.github.io/FIDASIM/ is my favourite example).

Personally, I find the FORD generated documentation easier to navigate than doxygen. In case we decide to use FORD we should agree on the symbols to indicate documentation and whether we use preceding documentation strings or not.

I am not sure if FORD can be used together with jin2for. Does one only document the generic interface block in that case?

cc: @cmacmackin

[cmacmackin]

@ivan-pi I've never used jin2for so I can't be certain. However, FORD does allow files to be preprocessed prior to it parsing for documentation, so that could work. You'd probably want to give a detailed explanation only in the generic blocks, but still document the arguments in each specific implementation.

[marshallward]

MOM6 uses Doxygen-style comments, but these are converted into reST files which are then passed onto Sphinx.

An example Doxygen-marked file: https://github.com/NOAA-GFDL/MOM6/blob/dev/gfdl/src/parameterizations/vertical/MOM_vert_friction.F90

This uses the !<and !! syntax for marking up variables, subroutines, etc.

The Doxygen XML is then converted to reST by a local tool developed by a former visitor (itself forked from another project): https://github.com/angus-g/sphinxcontrib-autodoc_doxygen

although we are looking to migrate to this tool: https://github.com/vovkos/doxyrest

Once it's been converted to reST, then we just run it through the usual Sphinx process.

This is the generated output: https://mom6.readthedocs.io/en/dev-gfdl/api/generated/modules/mom_vert_friction.html#f/mom_vert_friction

Some limitations of the above system are that the Doxygen -> reST process depends on an unsupported tool which only exposes a fraction of the Doxygen content, and that there are apparently bugs inside of Doxygen's Fortran support which prevent adoption of doxyrest (which is a supported tool). We have someone here working on this problem, but it's not ready to go at this moment.

Not necessarily advocating this approach BTW, only hoping to show what alternatives are out there and what is possible.

I agree with others that the native Doxygen output is not that pleasant to navigate, and I don't think we'd continue to use it if we didn't have the pipeline above.

[zbeekman]

I am not sure if FORD can be used together with jin2for. Does one only document the generic interface block in that case?

I usually generate the sources with jin2for then run FORD on those.

[zbeekman]

I do quite like Sphinx but the Doxygen to Sphinx pipeline seems complicated. Ideally whichever tool we choose will be Fortran aware enough to provide additional useful information beyond the source file markup.

For high-level documentation I really like read-the-docs and using mkdocs. E.g., https://zstdfortranlib.readthedocs.io/en/latest/?badge=latest (WIP)

[jvdp1]

I think it would be a good time to introduce a documentation.
Several modules are now present in stdlib, and it would be useful to have some documentations to show what is already implemented to the new contributors (and users).

From the different comments, it seems that FORD could be a good tool for generating a doc from Fortran files, while @marshallward and @zbeekman proposed more complicated (better?) approaches. Should we give a try to FORD first?

[certik]

Does FORD support Markdown? I would suggest to write our documentation in Markdown in the comments. We can start with FORD.

On Sun, Jan 5, 2020, at 7:11 AM, Jeremie Vandenplas wrote:

I think it would be a good time to introduce a documentation.
Several modules are now present in stdlib, and it would be useful to
have some documentations to show what is already implemented to the new
contributors (and users).

From the different comments, it seems that FORD could be a good tool
for generating a doc from Fortran files, while @marshallward
https://github.com/marshallward and @zbeekman
https://github.com/zbeekman proposed more complicated (better?)
approaches. Should we give a try to FORD first?

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/fortran-lang/stdlib/issues/4?email_source=notifications&email_token=AAAFAWHO6Q6NIKB2GYF4HRDQ4HTCJA5CNFSM4J25GCIKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEIDX2EA#issuecomment-570916112, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAAFAWC364JQWTTJ3GEGGDDQ4HTCJANCNFSM4J25GCIA.

[cmacmackin]

Yes, ford is markdown based.

On Sun, 5 Jan 2020, 17:22 Ondřej Čertík, [e-mail] wrote:

Does FORD support Markdown? I would suggest to write our documentation in
Markdown in the comments. We can start with FORD.

On Sun, Jan 5, 2020, at 7:11 AM, Jeremie Vandenplas wrote:

I think it would be a good time to introduce a documentation.
Several modules are now present in stdlib, and it would be useful to
have some documentations to show what is already implemented to the new
contributors (and users).

From the different comments, it seems that FORD could be a good tool
for generating a doc from Fortran files, while @marshallward
https://github.com/marshallward and @zbeekman
https://github.com/zbeekman proposed more complicated (better?)
approaches. Should we give a try to FORD first?

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
<
https://github.com/fortran-lang/stdlib/issues/4?email_source=notifications&email_token=AAAFAWHO6Q6NIKB2GYF4HRDQ4HTCJA5CNFSM4J25GCIKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEIDX2EA#issuecomment-570916112>,
or unsubscribe <
https://github.com/notifications/unsubscribe-auth/AAAFAWC364JQWTTJ3GEGGDDQ4HTCJANCNFSM4J25GCIA
.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/fortran-lang/stdlib/issues/4?email_source=notifications&email_token=AB6ESPIXRLGF4TSO32Z542TQ4IJOJA5CNFSM4J25GCIKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEID3MKQ#issuecomment-570930730,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AB6ESPMT7OMOU33PISZVEA3Q4IJOJANCNFSM4J25GCIA
.

[certik]

Perfect. I would give it a try.

On Sun, Jan 5, 2020, at 10:35 AM, Chris MacMackin wrote:

Yes, ford is markdown based.

On Sun, 5 Jan 2020, 17:22 Ondřej Čertík, [e-mail] wrote:

Does FORD support Markdown? I would suggest to write our
documentation in
Markdown in the comments. We can start with FORD.

On Sun, Jan 5, 2020, at 7:11 AM, Jeremie Vandenplas wrote:

I think it would be a good time to introduce a documentation.
Several modules are now present in stdlib, and it would be
useful to
have some documentations to show what is already implemented to
the new
contributors (and users).

From the different comments, it seems that FORD could be a good
tool
for generating a doc from Fortran files, while @marshallward
https://github.com/marshallward and @zbeekman
https://github.com/zbeekman proposed more complicated (better?)
approaches. Should we give a try to FORD first?

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
<

https://github.com/fortran-lang/stdlib/issues/4?email_source=notifications&email_token=AAAFAWHO6Q6NIKB2GYF4HRDQ4HTCJA5CNFSM4J25GCIKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEIDX2EA#issuecomment-570916112>,
or unsubscribe <

https://github.com/notifications/unsubscribe-auth/AAAFAWC364JQWTTJ3GEGGDDQ4HTCJANCNFSM4J25GCIA

.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub

https://github.com/fortran-lang/stdlib/issues/4?email_source=notifications&email_token=AB6ESPIXRLGF4TSO32Z542TQ4IJOJA5CNFSM4J25GCIKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEID3MKQ#issuecomment-570930730,
or unsubscribe

https://github.com/notifications/unsubscribe-auth/AB6ESPMT7OMOU33PISZVEA3Q4IJOJANCNFSM4J25GCIA
.

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/fortran-lang/stdlib/issues/4?email_source=notifications&email_token=AAAFAWC5O6NOOW5BCLBZUFLQ4IK43A5CNFSM4J25GCIKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEID3T4I#issuecomment-570931697, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAAFAWDBOD4EJHKATAA3TB3Q4IK43ANCNFSM4J25GCIA.

[zbeekman]

A question for @milancurcic and @certik: Should we host FORD API documentation with github pages? If so then I suggest we pick one of the following approaches for hosting and then deploy the documentation during CI.

Approach 1

Publish API documentation as part of this repository using an orphan gh-pages branch. This would replace the readme at http://fortran-lang.org with the FORD documentation, but the landing page could include the readme.

Approach 2

Create a new github repo to host the FORD API documentation and tweak the DNS settings for fortran-lang.org to point something like api-docs.fortran-lang.org to the documentation published in the other repository. I'd be happy to lead administering that documentation repo and help to setup a GitHub action to publish with FORD.

[certik]

I would do the second approach and use a url docs.fortran-lang.org. That way we can keep the landing page separate from the documentation. If you could set it up, that would be awesome.

[milancurcic]

I think approach 2 is better if we're considering landing page for fortran-lang to be its own thing eventually (separate from stdlib). I think we should as the fortran-lang website should encompass more than just stdlib -- eventually fpm, getting started, resources etc.

As for the stdlib landing page, it makes sense to me that it goes together with the documentation. For example, FORD docs for wavy include the README as the landing page: https://wavebitscientific.github.io/wavy/

If we make the Org (fortran-lang) GH page to be fortran-lang.org, then the GH page for stdlib will automatically become fortran-lang.org/stdlib, which I think is the most meaningful and intuitive url. docs.fortran-lang.org would make sense at the time being, but would be confusing when fortran-lang.org transcends stdlib.

[milancurcic]

Sorry, on re-reading Zaak's question and my response, I'd like to take that back.

If we agree that stdlib landing page + docs make sense as one website (I think they do, see the wavy example; FORD just tacks on documentation pages and styles it, but you still need a meaningful landing page for the docs), then I would do FORD docs as a GH site in the existing repo (stdlib), just as the current website is. So this is approach 1.

When we need other websites, they can go in their own repos:

• fortran-lang home page as Org-level GH website;
• fpm website and docs in the fpm repo;
• and so on

[milancurcic]

Publish API documentation as part of this repository using an orphan gh-pages branch

Alternative (and my preference) to gh-pages branch is to have a docs/ directory in master.

[certik]

I would not put any auto generated files (such as FORD docs) into the stdlib repository, otherwise everybody has to clone it all the time, increasing the download traffic and download time. I would definitely not put it into the docs/ directory into the master branch, but I would not even put it into the gh-pages branch (because it still gets cloned, and forked, etc.). Instead, let the stdlib repository CI generate the docs, and push them to a separate repository. That separate repository will then get hosted at some url online (we can discuss where).

[milancurcic]

I haven't considered size. wavy docs/ is 21M, which is quite heavy, and stdlib will be bigger than this. I think it's important that stdlib repo is small and lean on disk. I think it's okay to use a separate repo, even if less elegant approach.

[zbeekman]

OK, I think the 3 of us are in agreement that we'll put the FORD API docs into a separate repository. If this is true, what should we call the repository (on github, separate from what the URL is)? If we create an org page later, then I think something like fortran-lang/stdlib-docs, fortran-lang/stdlib-API-docs or similar would be the best bet because it will be accessible at fortran-lang.org/stdlib-docs. For the near-term a dns entry like api.fortran-lang.org or stdlib-docs.fortran-lang.org etc. would be a good place to start.

FYI I don't have sufficient org permissions to create a repo under the fortran-lang organization. So if we can pick a good name and then someone create it that would be great. Having owner or admin perms on the documentation repo might be needed/helpful, but I can try to do what I can with push only access and see if I run into any issues. I'd also be happy to help out with DNS provider stuff etc etc but figuring out credentials and permissions/roles can be tricky so I'll defer to you guys.

[milancurcic]

Okay, I thought Org members can create new repos but maybe not. Ondrej or I can create a repo and set adequate permissions at the repo level.

For the repo name, I prefer stdlib-docs over stdlib-API-docs (redundant).

We can worry about subdomains and DNS stuff later. I own the domain so I can manage that end.

[certik]

Let's go with stdlib-docs. We can always rename it I think. If you agree, I'll create it and set the permissions so that @zbeekman can take it from there.

[milancurcic]

Perfect, let's do it.

[certik]

Here you go, and @zbeekman is the admin for it:

https://github.com/fortran-lang/stdlib-docs

so you should be able to take it from here.

[leonfoks]

I've had great success with Ford. Markdown makes it super easy to include code snippets to help a user.

I might have missed this being stated somewhere, but is there a strict requirement for documentation of all functions and subroutines. If not I think that should be added to the style guide? or somewhere else. Having documentation debt is worse than having to use MPI in R.

Once the tool is decided on, a set of example style guides for functions, subs, interfaces, derived types, modules and submodules would be nice to follow. (I can help with this if needed)

Finally, having docs for the arguments of functions is nice, but having usability code snippets really helps too. Something like this which produces these docs using Ford. This is also an example of when to ignore the dependency diagram :P

[zbeekman]

Once the tool is decided on, a set of example style guides for functions, subs, interfaces, derived types, modules and submodules would be nice to follow. (I can help with this if needed)

This would be great. As a place to start collecting this, we could create a page (or pages) on the wiki while we get automated documentation bootstrapped and deployed. Then, since it's markdown, we can migrate it to FORD. I think FORD also has a warning flag to catch un-documented entities that we could integrate into the CI process to try to ensure properly documented code.

Finally, having docs for the arguments of functions is nice, but having usability code snippets really helps too.

I agree. I just helped a colleague figure out how to grab items from an array of json objects with JSON Fortran. There is great API documentation in that great project (I may be biased here...) but there are not as many examples as there could be. Or at least easy to find examples.

[jvdp1]

Found this issue back. Is it any agreements on this topic?

[certik]

@zbeekman do you still plan to get it setup https://github.com/fortran-lang/stdlib-docs? If you got busy, let us know, there is no problem. I am just asking so that we do not duplicate your work.

@jvdp1 I think we all agree we need documentation. And regarding the details, who ever is pushing this should simply choose something, get started, and then if something better comes along, we can switch. It seems FORD is the only documentation tool currently that allows to extract comments, so it might be the only option, unless we want to implement something from scratch, which I would recommend not to do initially.

[zbeekman]

Yes, I got busy, but just migrated documentation deployment for a work
project to a de GitHub actions and should be able to take a crack at this
tonight or later this week.

On Mon, Mar 30, 2020 at 5:14 PM Ondřej Čertík [e-mail]
wrote:

@zbeekman https://github.com/zbeekman do you still plan to get it setup
https://github.com/fortran-lang/stdlib-docs? If you got busy, let us
know, there is no problem. I am just asking so that we do not duplicate
your work.

@jvdp1 https://github.com/jvdp1 I think we all agree we need
documentation. And regarding the details, who ever is pushing this should
simply choose something, get started, and then if something better comes
along, we can switch. It seems FORD is the only documentation tool
currently that allows to extract comments, so it might be the only option,
unless we want to implement something from scratch, which I would recommend
not to do initially.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/fortran-lang/stdlib/issues/4#issuecomment-606253546,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AACEIPGVVNN62SFZPTSUKB3RKEDUPANCNFSM4J25GCIA
.

[certik]

@zbeekman that would be awesome! We'll wait.

[milancurcic]

@zbeekman Let us know if you need any help with this.

[zbeekman]

Should this be closed now that #183 is merged, or should we keep it open?

[milancurcic]

We're far from done. This should stay open.