Docfx: MSBuild errors when generating metadata for SDK based net461 project and VS 2017 15.6

[GeorgDangl]

DocFX Version Used: 2.32.0

Template used: default with an additional custom css file

Steps to Reproduce:

1. Checkout the repro project
   The project uses net461 as target framework.
2. Have at least Visual Studio 2017 15.6 installed (same issue with 15.6 Preview 7.0)
3. Run powershell ./build.ps1 BuildDocFxMetadata at the solution root

Expected Behavior:

Generate no warnings.

Actual Behavior:

Unclear MSBuild warnings are present in the output:

> C:\Users\Georg\.nuget\packages\docfx.console\2.32.0\tools\docfx.exe metadata "D:\Visual Studio Projects\Nuke.WebDeploy\docfx.json" --logLevel Verbose
[18-03-06 04:54:21.659]Verbose:Config file "D:\Visual Studio Projects\Nuke.WebDeploy\docfx.json" is used.
[18-03-06 04:54:21.697]Info:[ExtractMetadata]Using msbuild E:\Visual Studio 2017\MSBuild\15.0\Bin as inner compiler.
[18-03-06 04:54:21.782]Info:[ExtractMetadata]Loading projects...
[18-03-06 04:54:21.782]Verbose:[ExtractMetadata](D:/Visual Studio Projects/Nuke.WebDeploy/src/Nuke.WebDeploy/Nuke.WebDeploy.csproj)Loading project...
[18-03-06 04:54:22.792]Warning:[ExtractMetadata](D:/Visual Studio Projects/Nuke.WebDeploy/src/Nuke.WebDeploy/Nuke.WebDeploy.csproj)Workspace failed with: [Failure] Msbuild failed when processing the file 'D:\Visual Studio Projects\Nuke.WebDeploy\src\Nuke.WebDeploy\Nuke.WebDeploy.csproj' with message: MSB0001: Internal MSBuild Error: Missing resource ''
[18-03-06 04:54:22.903]Verbose:[ExtractMetadata](D:/Visual Studio Projects/Nuke.WebDeploy/src/Nuke.WebDeploy/Nuke.WebDeploy.csproj)Project D:\Visual Studio Projects\Nuke.WebDeploy\src\Nuke.WebDeploy\Nuke.WebDeploy.csproj loaded.
[18-03-06 04:54:22.903]Warning:[ExtractMetadata]Project 'D:\Visual Studio Projects\Nuke.WebDeploy\src\Nuke.WebDeploy\Nuke.WebDeploy.csproj' does not contain any documents.
[18-03-06 04:54:22.903]Info:[ExtractMetadata]Generating metadata for each project...
[18-03-06 04:54:23.263]Verbose:[ExtractMetadata]Successfully generated metadata D:/Visual Studio Projects/Nuke.WebDeploy/src/Nuke.WebDeploy/obj/xdoc/cache/obj for Nuke.WebDeploy
[18-03-06 04:54:23.274]Verbose:[ExtractMetadata]Completed MergeMetadata in 1,9675 milliseconds.
[18-03-06 04:54:23.274]Verbose:[ExtractMetadata]Completed MergeReference in 0,7387 milliseconds.
[18-03-06 04:54:23.274]Warning:[ExtractMetadata]No metadata is generated for Nuke.WebDeploy.
[18-03-06 04:54:23.274]Info:Completed in 1598,5294 milliseconds


Build succeeded with warning.
[18-03-06 04:54:23.290]Warning:[ExtractMetadata](D:/Visual Studio Projects/Nuke.WebDeploy/src/Nuke.WebDeploy/Nuke.WebDeploy.csproj)Workspace failed with: [Failure] Msbuild failed when processing the file 'D:\Visual Studio Projects\Nuke.WebDeploy\src\Nuke.WebDeploy\Nuke.WebDeploy.csproj' with message: MSB0001: Internal MSBuild Error: Missing resource ''
[18-03-06 04:54:23.290]Warning:[ExtractMetadata]Project 'D:\Visual Studio Projects\Nuke.WebDeploy\src\Nuke.WebDeploy\Nuke.WebDeploy.csproj' does not contain any documents.
[18-03-06 04:54:23.290]Warning:[ExtractMetadata]No metadata is generated for Nuke.WebDeploy.
        3 Warning(s)
        0 Error(s)

The interesting bit is:
MSB0001: Internal MSBuild Error: Missing resource ''
That doesn't say a lot about what is wrong and I was unable to find any cause for it.

Additional information

The metadata seems to be generated correctly, it puts files in the /api folder and the documentation looks correct: https://docs.dangl-it.com/Projects/Nuke.WebDeploy/1.0.1-beta0003/index.html

docfx.json specifies metadata.src.properties.TargetFramework as net461.

Manually invoking MSBuild.exe <path-to-csproj> compiles the app just fine, as does dotnet build with SDK 2.1.4 as well as the latest preview 2.1.300-preview1-008174.

The same happens for netstandard and netcoreapp projects, but here I can mitigate it by setting the MSBUILD_EXE_PATH to the one from the dotnet SDK:

// So it uses a fixed, known version of MsBuild to generate the metadata. Otherwise,
// updates of dotnet or Visual Studio could introduce incompatibilities and generation failures
var dotnetPath = Path.GetDirectoryName(ToolPathResolver.GetPathExecutable("dotnet.exe"));
var msBuildPath = Path.Combine(dotnetPath, "sdk", DocFxDotNetSdkVersion, "MSBuild.dll");
SetVariable("MSBUILD_EXE_PATH", msBuildPath);

The error does not happen on my build server, on which I have Visual Studio 2017 15.5.2
It logs that it uses MSBuild from the Visual Studio installation:

Info:[ExtractMetadata]Using msbuild C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\MSBuild\15.0\Bin as inner compiler.

I noticed the same error already yesterday evening on another machine in another project after updating Visual Studio 2017 to 15.6.

[vicancy]

How to sync with Microsoft.Build's latest update with little dev effort? Notification? Any ideas? @vwxyzh @superyyrrzz

[GeorgDangl]

I looked at your PR, @vicancy. Am I correct to assume that I can use the Microsoft.Build NuGet package and supply the Microsoft.Build.dll within as MSBUILD_EXE_PATH parameter to DocFX to use it?

[bitfringe]

The metadata seems to be generated correctly, it puts files in the /api folder and the documentation looks correct

For me the metadata is not generated at all. Try deleting obj folder/cache to force it to regenerate metadata from scratch.
I couldnt get it to work at all.

Same MSBuild related errors after updating to VS 15.6

[jskeet]

Same problem here. Here's a tiny repro:

Class1.cs:

namespace VS156
{
    /// <summary>This is a class</summary>
    public class Class1
    {
    }
}

VS156.csproj:

<Project Sdk="Microsoft.NET.Sdk">
  <PropertyGroup>
    <TargetFramework>netstandard2.0</TargetFramework>
  </PropertyGroup>
</Project>

docfx.json:

{
  "metadata": [
    {
      "src": [ { "files": [ "VS156.csproj" ] } ],
      "dest": "obj/api"
    }
  ]
}

Output from running docfx 2.32.1:

[18-03-08 10:20:44.103]Info:Config file docfx.json found, start generating metadata...
[18-03-08 10:20:44.141]Info:[MetadataCommand.ExtractMetadata]Environment variable VSINSTALLDIR is set to C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\, it is used as the inner compiler.
[18-03-08 10:20:44.265]Info:[MetadataCommand.ExtractMetadata]Loading projects...
[18-03-08 10:20:45.544]Warning:[MetadataCommand.ExtractMetadata](c:/Users/jon/Test/Projects/Issues/docfx/VS156/VS156.csproj)Workspace failed with: [Failure] Msbuild failed when processing the file 'c:\Users\jon\Test\Projects\Issues\docfx\VS156\VS156.csproj' with message: MSB0001: Internal MSBuild Error: Missing resource ''
[18-03-08 10:20:45.696]Warning:[MetadataCommand.ExtractMetadata]Project 'c:\Users\jon\Test\Projects\Issues\docfx\VS156\VS156.csproj' does not contain any documents.
[18-03-08 10:20:45.723]Info:[MetadataCommand.ExtractMetadata]Cache for c:/Users/jon/Test/Projects/Issues/docfx/VS156/VS156.csproj in c:\Users\jon\Test\Projects\Issues\docfx\VS156\obj\xdoc\cache\final\320869301 is corrupted, rebuild...
[18-03-08 10:20:45.725]Info:[MetadataCommand.ExtractMetadata]Generating metadata for each project...
[18-03-08 10:20:45.996]Info:[MetadataCommand.ExtractMetadata]'c:/Users/jon/Test/Projects/Issues/docfx/VS156/VS156.csproj' keep up-to-date since '08/03/2018 10:18:31', cached intermediate result 'c:/Users/jon/Test/Projects/Issues/docfx/VS156/obj/xdoc/cache/obj\wuymkhsv.0b4' is used.
[18-03-08 10:20:46.140]Warning:[MetadataCommand.ExtractMetadata]No metadata is generated for VS156.
[18-03-08 10:20:46.157]Info:[MetadataCommand]Completed Scope:MetadataCommand in 2025.2006 milliseconds.
[18-03-08 10:20:46.158]Info:Completed in 2030.6871 milliseconds


Build succeeded with warning.

[davkean]

Looks like this: https://github.com/Microsoft/msbuild/issues/2997.

[vicancy]

docfx leverages MSBuildLocator to resolve installed VS instances however it seems to fail resolving the instances according to https://github.com/Microsoft/MSBuildLocator/issues/17.
As a workaround, please set the following env before build: (mine is Enterprise version)

set VSINSTALLDIR=C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise
set VisualStudioVersion=15.0
set MSBuildExtensionsPath=C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\MSBuild\15.0\Bin

[jskeet]

That workaround isn't working for me, unfortunately. I've got the Community edition and I've changed both of the paths accordingly, but I'm seeing the same issue :(

[rainersigwald]

This is indeed Microsoft/msbuild#2997, which results from a mismatch between the Microsoft.Build.dll version and VS.

You can get the effects of #2494 without waiting for a docfx release by copying the VS-delivered MSBuild assemblies into the docfx folder next to docfx.exe, overwriting what's there. That worked for me against VS 15.6.1.

S:\work\docfx_locator>docfx\docfx.exe docfx.json
[18-03-09 07:45:15.306]Info:[MetadataCommand.ExtractMetadata]Using msbuild C:\Program Files (x86)\Microsoft Visual Studio\Preview\Enterprise\MSBuild\15.0\Bin as inner compiler.
[18-03-09 07:45:15.515]Info:[MetadataCommand.ExtractMetadata]Loading projects...
[18-03-09 07:45:17.584]Warning:[MetadataCommand.ExtractMetadata](S:/work/docfx_locator/VS156.csproj)Workspace failed with: [Failure] Msbuild failed when processing the file 'S:\work\docfx_locator\VS156.csproj' with message: MSB0001: Internal MSBuild Error: Missing resource ''
[18-03-09 07:45:17.816]Warning:[MetadataCommand.ExtractMetadata]Project 'S:\work\docfx_locator\VS156.csproj' does not contain any documents.
[18-03-09 07:45:17.855]Info:[MetadataCommand.ExtractMetadata]Cache for S:/work/docfx_locator/VS156.csproj in S:\work\docfx_locator\obj\xdoc\cache\final\-691546932 is corrupted, rebuild...
[18-03-09 07:45:17.861]Info:[MetadataCommand.ExtractMetadata]Generating metadata for each project...
[18-03-09 07:45:19.183]Warning:[MetadataCommand.ExtractMetadata]No metadata is generated for VS156.
[18-03-09 07:45:19.196]Info:[MetadataCommand]Completed Scope:MetadataCommand in 3929.5966 milliseconds.
[18-03-09 07:45:19.198]Info:Completed in 3938.355 milliseconds


Build succeeded with warning.
[18-03-09 07:45:19.220]Warning:[MetadataCommand.ExtractMetadata](S:/work/docfx_locator/VS156.csproj)Workspace failed with: [Failure] Msbuild failed when processing the file 'S:\work\docfx_locator\VS156.csproj' with message: MSB0001: Internal MSBuild Error: Missing resource ''
[18-03-09 07:45:19.237]Warning:[MetadataCommand.ExtractMetadata]Project 'S:\work\docfx_locator\VS156.csproj' does not contain any documents.
[18-03-09 07:45:19.251]Warning:[MetadataCommand.ExtractMetadata]No metadata is generated for VS156.
        3 Warning(s)
        0 Error(s)

S:\work\docfx_locator>copy "C:\Program Files (x86)\Microsoft Visual Studio\Preview\Enterprise\MSBuild\15.0\Bin\Microsoft.Build*dll" docfx
C:\Program Files (x86)\Microsoft Visual Studio\Preview\Enterprise\MSBuild\15.0\Bin\Microsoft.Build.Conversion.Core.dll
C:\Program Files (x86)\Microsoft Visual Studio\Preview\Enterprise\MSBuild\15.0\Bin\Microsoft.Build.dll
Overwrite docfx\Microsoft.Build.dll? (Yes/No/All): a
C:\Program Files (x86)\Microsoft Visual Studio\Preview\Enterprise\MSBuild\15.0\Bin\Microsoft.Build.Engine.dll
C:\Program Files (x86)\Microsoft Visual Studio\Preview\Enterprise\MSBuild\15.0\Bin\Microsoft.Build.Framework.dll
C:\Program Files (x86)\Microsoft Visual Studio\Preview\Enterprise\MSBuild\15.0\Bin\Microsoft.Build.Tasks.Core.dll
C:\Program Files (x86)\Microsoft Visual Studio\Preview\Enterprise\MSBuild\15.0\Bin\Microsoft.Build.Utilities.Core.dll
        6 file(s) copied.

S:\work\docfx_locator>docfx\docfx.exe docfx.json
[18-03-09 07:49:25.930]Info:[MetadataCommand.ExtractMetadata]Using msbuild C:\Program Files (x86)\Microsoft Visual Studio\Preview\Enterprise\MSBuild\15.0\Bin as inner compiler.
[18-03-09 07:49:26.157]Info:[MetadataCommand.ExtractMetadata]Loading projects...
[18-03-09 07:49:30.869]Info:[MetadataCommand.ExtractMetadata]Cache for S:/work/docfx_locator/VS156.csproj in S:\work\docfx_locator\obj\xdoc\cache\final\-691546932 is corrupted, rebuild...
[18-03-09 07:49:30.882]Info:[MetadataCommand.ExtractMetadata]Generating metadata for each project...
[18-03-09 07:49:34.560]Info:[MetadataCommand]Completed Scope:MetadataCommand in 8683.3688 milliseconds.
[18-03-09 07:49:34.574]Info:Completed in 8703.1626 milliseconds


Build succeeded.
        0 Warning(s)
        0 Error(s)

Having to update the MSBuild reference isn't ideal and shouldn't be required; that's what MSBuildLocator is for. But it looks like Locator + Roslyn MSBuildWorkspace may not be a very happy pairing yet. We'll need to track down and fix that, @andygerlicher.

[jskeet]

@rainersigwald: Fabulous, thanks - I can confirm that workaround works for me.

[MariusVolkhart]

To piggyback on the workaround, for those on Mac using a recent version of Mono, you'll need to copy /Library/Frameworks/Mono.framework/Versions/Current/lib/mono/msbuild/15.0/bin/Microsoft.Build.dll

[ChrisChinchilla]

@MariusVolkhart Strangely this workaround has now stopped working for me, is it still working for you? I was using the Homebrew version of mono, but it definitely worked last time I tried.

[MariusVolkhart]

@ChrisChinchilla yes, it is still working for me. I was using either mono 5.8 or 5.10 when I first copied the file, and have since switched to 5.12.

That said, the 2.38.1 release from yesterday works without any workaround on Mac with Mono 5.12

[ChrisChinchilla]

@MariusVolkhart Hmm, it only seems to work with the mono version downloaded from the mono website and not the homebrew version, which is great, but messier than I'd like. Hmm, I may find a solution to that, but in the meantime :)

[ChrisChinchilla]

Which is slightly annoying, as by default Docfx installed with Homebrew installs the homebrew version of mono if you don't have it installed…

[rainersigwald]

FYI @radical, some Mono/MSBuild confusion here ☝, possibly related to Homebrew packaging?