Sdk: Improve manpage infrastucture

[omajid]

The man pages for various commands (such as dotnet-build) indicate they are generated from some other document using a tool:

$ head -1 ./dotnet-sdk-2.0.0-preview2-006497/src/cli/Documentation/manpages/sdk/dotnet-compile.1
.\" Automatically generated by Pandoc 1.15.1

What is the process to update these docs? Where can I find that original source?

[richlander]

That's a great question. Everything should be in the repo somewhere.

/cc @livarcocc Do you know?

[nguerrera]

@blackdwarf handled this in the past.

It looks like this was generated from that but the generation logic doesn't seem to have been checked in and I can't find any docs on it. Also, the the markdown has since been deleted, and the man pages left behind look out of date too. :(

I think we need to clean this up and have only markdown checked in and run the generation during the build.

I'll change this bug's title to reflect that.

[omajid]

Thanks for digging into it!

I think we need to clean this up and have only markdown checked in and run the generation during the build.

Are we okay with a build-time dependency on pandoc?

[nguerrera]

I don't see a problem with that, but maybe I'm missing something. We could use different tooling and format too. Do you have a suggestion?

I see now that the man pages are badly out of date. They all refer to the project.json era. :(

[omajid]

Do you have a suggestion?

Pandoc actually sounds really good to me. Let me take a look at this and see what I can come up with.

[omajid]

Hm. It looks like tools like pandoc are not available for certain distributions that .NET Core targets, including CentOS and RHEL.

Taking a step back:

• Do the sources have to be in Markdown? Is an alternative - say xml, asciidoc or another format - acceptable?
• Do the man pages have to be generated during build? Like translations, could it be something that is stored in repo and updated with some regularity?

[nguerrera]

Do the sources have to be in Markdown? Is an alternative - say xml, asciidoc or another format - acceptable?

There is no hard requirement on Markdown. We can certainly look at alternatives.

Do the man pages have to be generated during build? Like translations, could it be something that is stored in repo and updated with some regularity?

I can live with checking in the generated content if we at least have a 1-step script to run to regenerate and that script is checked in to the repo.

I am generally averse to checking in things that are generated from source, but I'm open to discussing tradeoffs in this specific case.

[omajid]

It looks like asciidoc-based tools (a2x in particular) are available pretty much everywhere I looked:

• RHEL/CentOS: http://mirror.centos.org/centos/7/os/x86_64/Packages/asciidoc-8.6.8-5.el7.noarch.rpm
• Ubuntu: https://launchpad.net/ubuntu/trusty/+package/asciidoc
• Debian: https://packages.debian.org/buster/asciidoc-base
• Fedora: https://admin.fedoraproject.org/pkgdb/package/rpms/asciidoc/
• OpenSUSE: https://software.opensuse.org/download.html?project=Publishing&package=asciidoc

Would you be okay if I converted the current docs to asciidoc and then modified the build to generate the man pages at build time?

[omajid]

And here is what a non-complete port of dotnet-build.1 looks like: https://gist.github.com/omajid/e77859bb1dc83b98c72a6311f497035f. Almost as nice as markdown.

[omajid]

Also, it looks like what we want man pages to look like is already maintained here: https://docs.microsoft.com/en-us/dotnet/core/tools/dotnet-publish. Sources are https://github.com/dotnet/docs/tree/master/docs/core/tools

These documents are almost already in the format of man pages. Even the section names pretty much match up. Maybe it would be better to use those (somehow) rather than duplicating them. As we can already see, those docs are pretty well maintained while these man pages are completely out of date.

[nguerrera]

Thanks for the follow up on this. I think the ascii-doc format would work fine. I also agree it would be even better to automate building the manpages directly from the docs.microsoft.com content.

[omajid]

Looks like the docs folks closed the other issue :(

I am not really sure what to do now.

[ghost]

@nguerrera, @richlander, @Petermarcu, if hooking up pandoc script in CI is not possible for 2.1 (https://github.com/dotnet/docs/issues/2791#issuecomment-372707046), could we run pandoc script locally and send PR somewhere with all converted manpages for review? Maybe the auto-converted md files will require some manual changes.
It would be very helpful if 2.1 ships out with new manpages. Thanks!

[nguerrera]

Sure, we can accept a PR to dotnet/cli with the updated man pages. I am still wary of manual process, but at this point it seems we're not getting anywhere on automation. :(

[livarcocc]

Moving this to 2.1.4xx for the automation part.

[chrisd8088]

This is just a quick comment to say that I appreciate the effort to include updated Linux man pages in the distribution, and I hope full automation is possible (if it's not already in place).

While building dotnet packages for an obscure Linux distribution, I found that while the Fedora .rpm packages could be repackaged easily, they appeared to be lacking the man1 manual pages found in the Debian dotnet-sdk .deb packages.

So I ended up pulling the man1 pages out of the .deb package and into my custom package.

After a small amount of research, I wondered if this was due to the RHEL not having pandoc installed by default, or something like that (see also the comment above), or if it's just that the manual process bypassed these files for the Fedora .rpm packages (or if I'm just not finding them, but they are there in another Fedora package).