Roslyn: MSBuild parameter -p:RunAnalyzers=false is ignored

[foriequal0]

Version Used:

dotnet --info
.NET Core SDK(global.json 반영):
 Version:   3.1.100
 Commit:    cd82f021f4

런타임 환경:
 OS Name:     Windows
 OS Version:  10.0.18363
 OS Platform: Windows
 RID:         win10-x64
 Base Path:   C:\Program Files\dotnet\sdk\3.1.100\

Host (useful for support):
  Version: 3.1.0
  Commit:  65f04fb6db

.NET Core SDKs installed:
  3.1.100 [C:\Program Files\dotnet\sdk]

.NET Core runtimes installed:
  Microsoft.AspNetCore.All 2.1.14 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.App 2.1.14 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 3.1.0 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.NETCore.App 2.1.14 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 3.1.0 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.WindowsDesktop.App 3.1.0 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]

Steps to Reproduce:

1. Make a project

dotnet new console --name test-dotnet-run-analyzers

1. Add a bunch of analyzers

dotnet add package Microsoft.CodeAnalysis.FxCopAnalyzers
dotnet add package StyleCop.Analyzers

1. Build with -p:RunAnalyzers=false
   Korean characters are mixed in the build output but I think you can get the point.

dotnet build -p:RunAnalyzers=false
.NET Core용 Microsoft (R) Build Engine 버전 16.4.0+e901037fe
Copyright (C) Microsoft Corporation. All rights reserved.

  24.47 ms에서 C:\Users\user\workspace\test-dotnet-run-analyzers\test-dotnet-run-analyzers.csproj에 대한 복원을 완료했습니다.
Program.cs(3,11): warning CA1707: 네임스페이스 이름 'test_dotnet_run_analyzers'에서 밑줄을 제거하세요. [C:\Users\user\workspace\test-dotnet-run-analyzers\test-dotnet-run-analyzers.csproj]
Program.cs(1,1): warning SA1200: Using directive should appear within a namespace declaration [C:\Users\user\workspace\test-dotnet-run-analyzers\test-dotnet-run-analyzers.csproj]
Program.cs(3,11): warning SA1300: Element 'test_dotnet_run_analyzers' should begin with an uppercase letter [C:\Users\user\workspace\test-dotnet-run-analyzers\test-dotnet-run-analyzers.csproj]       Program.cs(5,11): warning SA1400: Element 'Program' should declare an access modifier [C:\Users\user\workspace\test-dotnet-run-analyzers\test-dotnet-run-analyzers.csproj]
Program.cs(7,21): warning SA1400: Element 'Main' should declare an access modifier [C:\Users\user\workspace\test-dotnet-run-analyzers\test-dotnet-run-analyzers.csproj]
Program.cs(9,31): warning CA1303: 'void Program.Main(string[] args)' 메서드가 리터럴 문자열을 'void Console.WriteLine(string value)' 호출의 'value' 매개 변수로 전달합니다. 대신 리소스 테이블에서 "Hello World!" 문자열을 가져오세요. [C:\Users\user\workspace\test-dotnet-run-analyzers\test-dotnet-run-analyzers.csproj]
Program.cs(7,35): warning CA1801: Main 메서드의 args 매개 변수가 사용되지 않았습니다. 매개 변수를 제거하거나 메서드 본문에 사용하세요. [C:\Users\user\workspace\test-dotnet-run-analyzers\test-dotnet-run-analyzers.csproj]
CSC : warning SA0001: XML comment analysis is disabled due to project configuration [C:\Users\user\workspace\test-dotnet-run-analyzers\test-dotnet-run-analyzers.csproj]
  test-dotnet-run-analyzers -> C:\Users\user\workspace\test-dotnet-run-analyzers\bin\Debug\netcoreapp3.1\test-dotnet-run-analyzers.dll

빌드했습니다.

Program.cs(3,11): warning CA1707: 네임스페이스 이름 'test_dotnet_run_analyzers'에서 밑줄을 제거하세요. [C:\Users\user\workspace\test-dotnet-run-analyzers\test-dotnet-run-analyzers.csproj]
Program.cs(1,1): warning SA1633: The file header is missing or not located at the top of the file. [C:\Users\user\workspace\test-dotnet-run-analyzers\test-dotnet-run-analyzers.csproj]
Program.cs(1,1): warning SA1200: Using directive should appear within a namespace declaration [C:\Users\user\workspace\test-dotnet-run-analyzers\test-dotnet-run-analyzers.csproj]
Program.cs(3,11): warning SA1300: Element 'test_dotnet_run_analyzers' should begin with an uppercase letter [C:\Users\user\workspace\test-dotnet-run-analyzers\test-dotnet-run-analyzers.csproj]       Program.cs(5,11): warning SA1400: Element 'Program' should declare an access modifier [C:\Users\user\workspace\test-dotnet-run-analyzers\test-dotnet-run-analyzers.csproj]
Program.cs(7,21): warning SA1400: Element 'Main' should declare an access modifier [C:\Users\user\workspace\test-dotnet-run-analyzers\test-dotnet-run-analyzers.csproj]
Program.cs(9,31): warning CA1303: 'void Program.Main(string[] args)' 메서드가 리터럴 문자열을 'void Console.WriteLine(string value)' 호출의 'value' 매개 변수로 전달합니다. 대신 리소스 테이블에서 "Hello World!" 문자열을 가져오세요. [C:\Users\user\workspace\test-dotnet-run-analyzers\test-dotnet-run-analyzers.csproj]
Program.cs(7,35): warning CA1801: Main 메서드의 args 매개 변수가 사용되지 않았습니다. 매개 변수를 제거하거나 메서드 본문에 사용하세요. [C:\Users\user\workspace\test-dotnet-run-analyzers\test-dotnet-run-analyzers.csproj]
CSC : warning SA0001: XML comment analysis is disabled due to project configuration [C:\Users\user\workspace\test-dotnet-run-analyzers\test-dotnet-run-analyzers.csproj]
    경고 9개

경과 시간: 00:00:00.90

1. false doesn't work too.

Expected Behavior:
Warnings should not be printed

Actual Behavior:
Warnings are printed. Even it can make MSBuild to return an error with -p:TreatWarningsAsErrors=true

Also, it seems that the build warnings are not cached. If I build twice, there are no warnings in the second build. The recent release of Rust's cargo (https://github.com/rust-lang/cargo/blob/master/CHANGELOG.md#added-2) added a feature that caches build warnings. (https://github.com/rust-lang/cargo/pull/7450). I prefer to separate the build step and the lint step for CI. dotnet clean between the steps won't be necessary if we cache warnings.

[jinujoseph]

cc @mavasani

[mavasani]

@foriequal0 Can you confirm if your project is importing Microsoft.CodeAnalysis.targets by running msbuild with -pp:out.xml and checking the contents of out.xml? RunAnalyzers MSBuild property is respected by Microsoft.CodeAnalysis.targets, and I suspect it is not working because it is not being imported.

You can also try to add the following to your targets file explicitly to get it to work:

<PropertyGroup>
    <IsCSharpOrVbProject Condition="'$(Language)'=='C#' or '$(Language)'=='VB'">true</IsCSharpOrVbProject>
    <RunAnalyzersDuringLiveAnalysis Condition="'$(RunAnalyzers)' != ''">$(RunAnalyzers)</RunAnalyzersDuringLiveAnalysis>
    <RunAnalyzersDuringLiveAnalysis Condition="'$(RunAnalyzersDuringLiveAnalysis)' == ''">$(IsCSharpOrVbProject)</RunAnalyzersDuringLiveAnalysis>
    <RunAnalyzersDuringBuild Condition="'$(RunAnalyzers)' != ''">$(RunAnalyzers)</RunAnalyzersDuringBuild>
    <RunAnalyzersDuringBuild Condition="'$(RunAnalyzersDuringBuild)' == ''">$(IsCSharpOrVbProject)</RunAnalyzersDuringBuild>
  </PropertyGroup>

  <Target Name="ComputeRunAnalyzers"
           BeforeTargets="CoreCompile"
           Condition="'$(RunAnalyzers)' == ''">
    <PropertyGroup>
      <IsAnyDesignTimeBuild Condition="'$(DesignTimeBuild)' == 'true' or '$(BuildingProject)' != 'true'">true</IsAnyDesignTimeBuild>
      <RunAnalyzers Condition="'$(IsAnyDesignTimeBuild)' == 'true' and '$(RunAnalyzersDuringLiveAnalysis)' == 'false'">false</RunAnalyzers>
      <RunAnalyzers Condition="'$(IsAnyDesignTimeBuild)' != 'true' and '$(RunAnalyzersDuringBuild)' == 'false'">false</RunAnalyzers>
    </PropertyGroup>
  </Target>

  <Target Name="DisableAnalyzers"
           AfterTargets="ComputeRunAnalyzers"
           Condition="'$(RunAnalyzers)' == 'false' and '$(IsAnyDesignTimeBuild)' != 'true'">
    <!-- 
       Disable analyzers via an MSBuild property settable on the command line. 
       Note that we remove analyzers completely only for command line builds. 
       For design time builds, IDE host should read this property and respect it for analyzer execution.
       Latter allows users to still view the available analyzers in Analyzers node and
       also execute analyzers on-demand with "Run Code Analysis" command.
     -->
    <ItemGroup>
      <Analyzer Remove="@(Analyzer)" />
    </ItemGroup>
  </Target>

[foriequal0]

@mavasani
Microsoft.CodeAnalysis.targets doesn't seem to be imported when I build it using dotnet build -pp:out.xml. Is it supposed to be used only in VisualStudio? I can't find RunAnalyzers at all, and this is the only relevant part with Microsoft.CodeAnalysis.target

  <PropertyGroup xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
    <CodeAnalysisTargets Condition="'$(CodeAnalysisTargets)'==''">$(MSBuildExtensionsPath32)\Microsoft\VisualStudio\v$(VisualStudioVersion)\CodeAnalysis\Microsoft.CodeAnalysis.targets</CodeAnalysisTargets>
  </PropertyGroup>
  <!--<Import Project="$(CodeAnalysisTargets)" Condition="Exists('$(CodeAnalysisTargets)')" />-->

Here's a zip archive of out.xml: out.zip

[mavasani]

Can you verify that file Microsoft.CodeAnalysis.Targets exists on your machine? It should be at $(MSBuildExtensionsPath32)\Microsoft\VisualStudio\v16.0\CodeAnalysis\Microsoft.CodeAnalysis.targets, MSBuildExtensionsPath32 is the MSBuild directory within your VS install directory.

[mavasani]

For example, on my machine, it is at C:\Program Files (x86)\Microsoft Visual Studio\2019\Preview\MSBuild\Microsoft\VisualStudio\v16.0\CodeAnalysis\Microsoft.CodeAnalysis.Targets

[foriequal0]

Yes I have it at C:\Program Files (x86)\Microsoft Visual Studio\2019\Professional\MSBuild\Microsoft\VisualStudio\v16.0\CodeAnalysis\Microsoft.CodeAnalysis.Targets

[foriequal0]

$(MSBuildExtensionPath32) is C:\Program Files (x86)\Microsoft Visual Studio\2019\Professional\MSBuild on VisualStudio Build, but C:\Program Files\dotnet\sdk\3.1.100 on dotnet build. There is no Microsoft\VisualStudio\v16.0\CodeAnalysis\Microsoft.CodeAnalysis.Targets after that.

[mavasani]

Seems to be a good reason to move the above target that I pasted into https://github.com/dotnet/roslyn/blob/master/src/Compilers/Core/MSBuildTask/Microsoft.Managed.Core.targets. I'll work on a PR. Until then, you can manually copy that MSBuild snippet into your targets file.

[foriequal0]

Great! Thank you! I'll be looking forward to it!
Can I get an answer to caching the warnings? Should I make another feature request issue?

[mavasani]

If I build twice, there are no warnings in the second build.

I think that issue is not specific to analyzer warnings. For example, consider the below code:

    public class Class1
    {
        void M()
        {
            var x = 0;
        }
    }

1. Build project. You should get a CS0219 warning.
2. Build project again - build succeeds without warnings. This is due to the fact that compiler inputs and compiler toolset is identical, so the build is skipped.

I think this is by-design for the compiler, but feel free to file a separate issue for the compiler team to triage.