Roslyn-analyzers: Question: How do i verify a code fix in a unit test for a diagnostic that isn't mine?

[taori]

I want to provide a codefix for CS0123 in my project and test the codefix - However the boilerplate code for unit tests expects an analyzer diagnostic and i can't seem to find a way to accomplish this.

Is this a scenario that is unwanted/impossible to write unit tests for or am i missing something?

[mavasani]

@taori Are you using Microsoft.CodeAnalysis.Testing NuGet package for analyzer/code fix test framework? Tagging @sharwell

[taori]

@mavasani
No. This is my project file.

https://github.com/taori/Amusoft.CodeAnalysis.Analyzers/blob/master/src/Amusoft.CodeAnalysis.Analyzers.Test/Amusoft.CodeAnalysis.Analyzers.Test.csproj

Is this possibly resolved for projects with a newer version of roslyn analyzer sdk projects?

[mavasani]

I don't think Microsoft.CodeAnalysis.Testing is still part of the roslyn analyzer SDK, but @jmarolf should confirm.

[sharwell]

The new test framework can definitely test this. I can send instructions next time I'm at a computer.

The library is not part of the analyzer SDK yet, but it's on MyGet and works with Roslyn 1.0.1 and newer.

[taori]

@sharwell is the name of this package Microsoft.CodeAnalysis.Testing as mavasani mentioned or is it different than that, if you can perhaps remember the name?

Also thanks for taking a look at this.

[sharwell]

https://github.com/dotnet/roslyn-sdk/blob/master/src/Microsoft.CodeAnalysis.Testing/README.md

[sharwell]

Keep in mind some things may have changed since that was written. The packages have the same name and location though.

[taori]

https://github.com/dotnet/roslyn-sdk/blob/master/src/Microsoft.CodeAnalysis.Testing/README.md

Thanks. I'll see how far i can get without the suggestions for now. I appreciate the pointer though :)

[taori]

@sharwell which packages should i add?
I've followed the guide at https://github.com/dotnet/roslyn-sdk/blob/master/src/Microsoft.CodeAnalysis.Testing/README.md

and added the package source, but i can't find a package named like

• Microsoft.CodeAnalysis.CSharp.Analyzer.Testing.XUnit
• Microsoft.CodeAnalysis.Testing

in the package source https://dotnet.myget.org/F/roslyn-analyzers/api/v3/index.json

are those the wrong package names or did they just change after the document was written?

[sharwell]

The approach I normally take is the following:

1. Set up the package source
2. Add a PackageReference for Microsoft.CodeAnalysis.CSharp.CodeFix.Testing.XUnit (and Microsoft.CodeAnalysis.VisualBasic.CodeFix.Testing.XUnit if you need to test Visual Basic)
3. Define a class CSharpCodeFixVerifier<TAnalyzer, TCodeFix> (and VisualBasicCodeFixVerifier<TAnalyzer, TCodeFix> if you need to test Visual Basic)
4. Define a nested class CSharpCodeFixVerifier<TAnalyzer, TCodeFix>.Test (likewise for Visual Basic)

Step 3 can be taken almost verbatim, except this code block is no longer required with the current releases of the testing library. Step 4 (the Test class) varies by project, but I recommend starting with an empty one:

public class Test : CSharpCodeFixTest<TAnalyzer, TCodeFix, XUnitVerifier>
{
}

[taori]

The approach I normally take is the following:

1. Set up the package source

2. Add a PackageReference for Microsoft.CodeAnalysis.CSharp.CodeFix.Testing.XUnit (and Microsoft.CodeAnalysis.VisualBasic.CodeFix.Testing.XUnit if you need to test Visual Basic)

3. Define a class [`CSharpCodeFixVerifier<TAnalyzer, TCodeFix>`](https://github.com/microsoft/vs-threading/blob/master/src/Microsoft.VisualStudio.Threading.Analyzers.Tests/Helpers/CSharpCodeFixVerifier%602.cs) (and `VisualBasicCodeFixVerifier<TAnalyzer, TCodeFix>` if you need to test Visual Basic)

4. Define a nested class [`CSharpCodeFixVerifier<TAnalyzer, TCodeFix>.Test`](https://github.com/microsoft/vs-threading/blob/master/src/Microsoft.VisualStudio.Threading.Analyzers.Tests/Helpers/CSharpCodeFixVerifier%602%2BTest.cs) (likewise for Visual Basic)

Step 3 can be taken almost verbatim, except this code block is no longer required with the current releases of the testing library. Step 4 (the Test class) varies by project, but I recommend starting with an empty one:

public class Test : CSharpCodeFixTest<TAnalyzer, TCodeFix, XUnitVerifier>
{
}

mhm yes, perhaps the package resource in the document is outdated? I am not getting those packages you mention. even with prerelease toggled on

[sharwell]

https://dotnet.myget.org/feed/roslyn-analyzers/package/nuget/Microsoft.CodeAnalysis.CSharp.CodeFix.Testing.XUnit

[taori]

Ugh. Picked the wrong package source. Sorry for wasting your time + Thanks for the help

[sharwell]

No problem, glad you got it working!

[taori]

@sharwell
Ok now that i have it working i still wonder how to implement the test - As far as i can tell from the API of Verifier it expects an analyzer still - but how can implement a test for a code fix where the diagnostic, which i want to fix is not my own (cs0123 e.g.).

I can detect the diagnostic itself - but it tells me this:

Mismatch between number of diagnostics returned, expected "1" actual "0"
which is odd, because if i change the column to 25 it is suddenly aware of a diagnostic.

I did not dig into sources yet, but it feels like the given diagnostics are then cleared of diagnostic id's which are not part of the Analyzer passed to the Verifier, which then raises the above exception.

Ideally this behavior would be customizable, because being able to provide fixes for foreign diagnostic id's would be quite an advantage

[sharwell]

The testing SDK provides EmptyDiagnosticAnalyzer for this case (and EmptyCodeFixProvider for cases where there is an analyzer but no fix).

[taori]

Oh boy, this new testing framework seems very convenient to work with i suppose. Should i raise a docs update issue to mention those two classes, or is that planned anyway once the SDK is being released?

I especially love the line indicators where code mismatches. Really enjoying this.

[sharwell]

Please feel free to file issues for anything you notice. 😄

[mavasani]

Closing this issue. @taori feel free to ping us if you need anything else.