Docfx: Support running docfx on .NET Core

[danroth27]

Today docfx runs on the full .NET Framework (and Mono). We should update docfx to run on .NET Core, as this will be the support xplat story for .NET console applications. Obviously this means working with the authors of the many third party dependencies that docfx has to get them onto .NET Core as well.

@blackdwarf

[agjohnson]

What is the current status for runtime support of docfx? Any ETA on coreclr support? I'm trying to run locally on OSX, a 1.0.0-rc1-update2 on a mono runtime works for the most part, though I hit exceptions with pieces that look to be Mono related.

[danroth27]

It's going to take a while to get docfx off of Mono and on to .NET Core. The problem is that docfx has a bunch of 3rd party dependencies that haven't been updated yet to work on .NET Core.

However, docfx did recently release a preview that can analyze .NET Core based projects: https://github.com/dotnet/docfx/releases/tag/1.9.0-alpha-0091-g1c1b291. I haven't had a chance to test it out yet though. Note that we are moving everything (including docfx) off of DNX.

[agjohnson]

Oh interesting, I didn't realize about the move from dnx -> dotnet cli tool. I should give a try with the latest mono and latest supporting toolchain. I don't know if that will clear up any of the bugs that I'm attributing to mono, but worth a shot.

[danroth27]

Yeah, moving off of DNX is a pretty big change for the product. The good news is that you no longer need to install DNX - docfx is now just a native executable.

[agc93]

Is there any update on this? DocFx seems to have almost ditched the xplat story:

• Commit 562f74be6bd33dfb02f0571c397f031907302ad1 and ef0632dd71661fbb191f800d50903a0811c35229 progressively removed most of the documentation references to the xplat version of DocFX
• This issue hasn't had an update in over a year
• Shipping as a Chocolatey package only works for Windows users
• The zip package bundled with each GitHub release includes only the exe
• .NET Core support has been RTM'ed for some time and 2.0 is just around the corner
• Many (looks like not all) of the dependencies are starting to support .NET Standard

Is there still plans to make this tool cross-platform? It was a big reason I chose it and at this point I'm looking at alternatives as support for .NET Core/xplat seems to still be a long way off.

[vicancy]

Hi @agc93 Migrating to net core is in our backlog however not yet with the highest priority. Currently mono docfx is working as an x-platform solution. Could you share your concerns about this solution? We can re-prioritize the issue if it is a common requirement from community.

[agc93]

@vicancy I'm currently building a .NET Core app on Linux and DocFX is the only thing that needs Mono: the app, build scripts, dependencies and server all run just fine on .NET Core natively, no Mono needed. Additionally, I can't CI build most of my projects as my CI environment doesn't include Mono (again .NET Core projects don't need it).

While I think using mono is an acceptable workaround I think .NET Core support still seems like a pretty important step given it's now been stable for some time and 2.0 is around the corner.

Realistically, I was using docfx to avoid extra dependencies, not add them

[vicancy]

@agc93 thanks for your feedback. Yes, I totally agree with you that .NET Core is a meaningful approach. We will investigate the effort migrating to .NET core. We have some 3rd party dependencies not supporting that, e.g. Microsoft.Owin, so it may take some effort to get rid of those dependencies.

[fritz-build]

@vicancy looks like owin is only used for docfx serve -- could just as easily use asp.net core with the static files middleware -- are there any other dependencies off the top of your head that need to be migrated? I'd be happy to contribute where needed -- this would make xplat much, much easier.

[vicancy]

@fritz-build One big change is our plugin system which uses another AppDomain. https://github.com/dotnet/docfx/blob/dev/src/docfx/SubCommands/BuildCommand.cs#L517

Honestly speaking, our team does not have enough resource currently to look into xplatform issues, It would be fantastic if you could contribute. We are eager for contributions and just let me know if I can do anything to help. From owin to asp.net core can be the first step out moving to netcore.

[dmccaffery]

Hey @vicancy i was logged in as @fritz-build by accident when I posted last. I’ll create an issue to convert to asp.net core and create an associated PR sometime this week. I appreciate that resources are limited; just thought that baby steps could go a long way.

[tugberkugurlu]

I imagine that effort to move this to .NET Core 2.0 will require less work considering that you can use most of the full .NET Framework dependencies and API surface are of .NET Standard is much larger.

[tugberkugurlu]

One big change is our plugin system which uses another AppDomain. https://github.com/dotnet/docfx/blob/dev/src/docfx/SubCommands/BuildCommand.cs#L517

@vicancy for example, AppDomain is part of .NET Standard 2.0 now: https://docs.microsoft.com/en-gb/dotnet/api/?view=netstandard-2.0&term=appdomain

[davidfowl]

You can't create app domains though.

[tugberkugurlu]

@davidfowl I am confused 😕 It says you can here: https://docs.microsoft.com/en-gb/dotnet/api/system.appdomain?view=netstandard-2.0 Am I looking at the wrong place?

[tugberkugurlu]

I am confused 😕 It says you can here: https://docs.microsoft.com/en-gb/dotnet/api/system.appdomain?view=netstandard-2.0 Am I looking at the wrong place?

meh, https://github.com/dotnet/corefx/blob/7a673547563dc92f1f4cad802d7b57483cdf7500/src/System.Runtime.Extensions/src/System/AppDomain.cs#L94

It would be off topic to ask why that API is on .NET Standard at the first place. So, I will find a better place to do it.

[davidfowl]

https://github.com/dotnet/standard/

[tugberkugurlu]

@davidfowl thx! Created https://github.com/dotnet/standard/issues/456

[tintoy]

I know I'm a little late to this party, but I'd like to add my vote for moving to .NET Core (or at least .NET Standard) at some point. I've been looking into redoing my DocFX extension to use an LSP language service for DocFX written in managed code (so I can reuse the DocFX engine libraries), but have been kinda blocked for a while now by the fact that some of those libraries have dependencies that aren't fully compatible with .NET Standard (not even 2.0). I could make a Windows-only language service but that sounds like no fun at all :-)

As for AppDomain, I've found there are plenty of more-cross-platform-friendly workarounds available (e.g. launching a separate process and communicating via NamedPipeClientStream / AnonymousPipeClientStream / STDIN+STDOUT).

I'm willing to put my hand up to spend some time / effort on helping migrate DocFX to .NET Standard / .NET Core (or work with anyone else if there are other volunteers).

[vicancy]

https://github.com/dotnet/docfx/issues/1963 to track the work items

[vicancy]

Thank you @tintoy It would be great if you can give a hand on this issue.

[dmccaffery]

Im 100% down. Let me know how I can help. Any guidance on how I can do so would be greatly appreciated.

[tintoy]

Wow, OK, yak-shaving time. There is no Mustache implementation currently compatible with .NET Standard (any version!). Looks like I'll have to create a .NET Standard port of Nustache (or similar).

Anyone who'd like to help - now's the time to pitch in :)

[tintoy]

Actually, let me see if it'll still run on .NET Standard 2.0 first (I get the usual NU1701 but it may still work correctly).

[agc93]

@tintoy Not strictly Mustache, but Handlebars.Net supports .NET Standard as a starting point

[sandorfr]

yes handlebars.nets support .Net standard for a long while now ( and I also have a fork of mustache-sharp since beta4 ;) https://www.nuget.org/packages/mustache-netstandard/ ). I would recommend Handlebars though.

[yufeih]

We are now entirely on .NET core in docfx v3, with tests for Windows/Mac/Linux. The app domain based plugin model is abandoned in v3, it may be replaced with separate process in the future.

[danroth27]

@yufeih Is there an ETA on the docfx v3 release?

[dmccaffery]

I LOVE YOU GUYS!

[yufeih]

@danroth27 it'll be at least the end of 2018, see details here

[Code-DJ]

@yufeih is there guidance on how to play with the v3 bits? If I were to create a project from scratch, what is the equivalent of "docfx init -q"? Thanks.

[yufeih]

@Code-DJ Right now there is no equivalent of docfx init.
To play with it, you need to manually create a folder structure like this.

Just to set expectation, the template system and C# API generation is not available yet.

[Romanx]

Hey, i'm the current maintainer of Nustache and it's successor Stubble.. I had some conversations with @tintoy around porting Nustache to netstandard and got caught up with some other things and so it fell off. In the time elapsed Stubble has reached maturity and i've been helping people migrate from Nustache since it's mainly only getting bug fixes now.

I just came to see if you'd like help using Stubble to migrate to dotnet core and it seems you've been very busy yourselves so congratulations. The v3 branch seems to have some Stubble code already there which could probably be a little optimised. Would you be interested in me helping at all?

[yufeih]

@Romanx , we would love to see you helping out ❤️ ❤️ ❤️

[jskeet]

Any updates on v3? It makes me sad that there's still no docfx release running on .NET Core :(

[superyyrrzz]

@jskeet On the way but not ready for public use now... See https://github.com/dotnet/docfx/blob/v3/docs/roadmap.md

[AndyPook]

@superyyrrzz wrt dotnet Core
I can't see any reference to dotnet Core in the roadmap.md
Is there an issue tracking this?

[superyyrrzz]

@AndyPook It is not mentioned in the roadmap, but v3 starts from .NET Core, and will always support it.

[AndyPook]

@superyyrrzz ok thanks for that. Is there a expected timeframe? I get limited resources etc 😄 not looking for a date, just a guide, weeks, months ?

My current solution is that I've created a docker base image with the mono etc deps installed and a separate CI step with a volume mount ...
https://gist.github.com/AndyPook/a36cfd8708800a4d3fdb06a1eb31eb83

[superyyrrzz]

@AndyPook Not within 2019. We do not determine the exact date, but possible in 2020.

I think there is some existing docfx image on docker hub that can make your work easier.

[roji]

Any timeline update on this? docfx is an important part of the .NET ecosystem and is used by many OSS projects out there - keeping this a modern, functional building block is really important.

[yufeih]

@roji There are two key pieces missing for the initial v3 community release: a community template and exposing the .NET API reference build. We've just started prototyping the community template here and will likely look at .NET API reference after that. Hopefully some early preview releases will come this calendar year.

[KalleOlaviNiemitalo]

I tried DocFX 3.0.0-beta1.246+8b9773b7f0 and https://github.com/docascode/default-template 1e3e3a9b33f2a1dfd5653efd4dac85b7e8125439 on a document tree that contains only Markdown documents, no .NET API. Installed as a .NET local tool on Windows. Found problems:

• dotnet docfx build --template docfx/default-template tries to open docfx\default-template\ContentTemplate\docfx\default-template\ContentTemplate\Conceptual.mta.json.js which does not exist. This path doubled the docfx\default-template\ContentTemplate part. No problem if I specify an absolute path.
• DocFX did not automatically copy the fonts, styles, and svg directories from the template to _site.
• Each Markdown file now translates to a directory with index.html in it, but docfx serve in DocFX 2.45.1 does not find that index.html if the directory name contains spaces or non-ASCII characters. (DocFX v3 doesn't seem to have docfx serve at all.) Filed https://github.com/aspnet/AspNetKatana/issues/371.
• DocFX displayed "Copyright © Microsoft." in the page footer even though I had set build.globalMetadata._appFooter in docfx.json. It seems _includes/footer.html.liquid does not receive the value of _appFooter. I also tried setting _appFooter in a YAML header in a Markdown file, and that had no effect either.