Coverlet: Coverage Incorrect for Async Method using Linq

[zmhh]

Latest of everything:

- netcoreapp3.1
- coverlet.collector 1.2.0 / coverlet.msbuild 2.8.0 (both have the same issue)
- Microsoft.NET.Test.Sdk 16.5.0
- NUnit 3.12.0
- NUnit3TestAdapter 13.16.1

If an async method uses Linq, coverage for all lines except the linq line(s) will be missing. Thus all other lines in the method are implied as uncoverable.

The following solution reproduces the issue and has the output file which shows the issue.

CoverletExample.zip

To reproduce, perform the following command:

dotnet test ExampleTest.csproj --configuration Release --collect:"XPlat Code Coverage"

The class in question:

public class ExampleClass
    {
        public async Task DoSomethingAsync(IEnumerable<object> objects)
        {
            await Task.Delay(TimeSpan.FromMilliseconds(1));
            foreach (var o in objects)
            {
                Console.WriteLine(o);
            }
        }

        public async Task DoSomethingAsyncWithLinq(IEnumerable<object> objects)
        {
            await Task.Delay(TimeSpan.FromMilliseconds(1));
            var selected = objects.Select(o => o);
            foreach (var o in selected)
            {
                Console.WriteLine(o);
            }
        }

        public void DoSomethingSync(IEnumerable<object> objects)
        {
            foreach (var o in objects)
            {
                Console.WriteLine(o);
            }
        }

        public void DoSomethingSyncWithLinq(IEnumerable<object> objects)
        {
            var selected = objects.Select(o => o);
            foreach (var o in selected)
            {
                Console.WriteLine(o);
            }
        }

Each method has a basic test passing it 1000 objects. Note that in the output all lines except for those in the method "DoSomethingAsyncWithLinq" are correctly reported. For the method "DoSomethingAsyncWithLinq", only the Linq line var selected = objects.Select(o => o); is reported:

<class name="Example.ExampleClass/&lt;&gt;c" filename="ExampleClass.cs" line-rate="1" branch-rate="1" complexity="2">
          <methods>
            <method name="&lt;DoSomethingAsyncWithLinq&gt;b__1_0" signature="(System.Object)" line-rate="1" branch-rate="1">
              <lines>
                <line number="22" hits="1001" branch="True" condition-coverage="100% (2/2)">
                  <conditions>
                    <condition number="126" type="jump" coverage="100%" />
                  </conditions>
                </line>
              </lines>
            </method>
          </methods>
          <lines>
            <line number="22" hits="1001" branch="True" condition-coverage="100% (2/2)">
              <conditions>
                <condition number="126" type="jump" coverage="100%" />
              </conditions>
            </line>
          </lines>
        </class>

[zmhh]

Also, in all cases the linq statement is listed to have been hit an equivalent number of times to the number of objects (or number of objects + 1). In the cases where the foreach is covered, the foreach statement is listed as being hit double that number of times. Not sure if this is correct behavior. But this is not a big deal for us as we don't really care about hit numbers.

[MarcoRossignoli]

Thank's for reporting this, we'll come back asap.

[MarcoRossignoli]

cc @matteoerigozzi

[MarcoRossignoli]

@zmhh before starting with investigation can you try with nightly build?We did some fix on master for async state machines https://github.com/tonerdo/coverlet/blob/master/Documentation/ConsumeNightlyBuild.md

[zmhh]

The output for the example solution was exactly the same for nightly version 1.2.29-gd2f0fd2248.

[MarcoRossignoli]

Ok thank a lot for tests we'll take a look asap

[MarcoRossignoli]

I confirm the strange behaviour, seem a bug.

[zmhh]

The same issue occurs in Debug configuration, just fyi.

[MarcoRossignoli]

yep @zmhh it's unrelated

[ravinsp]

Hi, we encountered similar issue after upgrading to coverlet.msbuild 2.8. Coverage being shown as partial for await statements. (similar behaviour with .Net core 2.2 and 3.1). We resolved it by downgrading back to coverlet.msbuild 2.7 which we were previously using without issues.
(Thanks for this great project btw)

[agtenr]

We are also encountering this issue.
2.7 seems to work fine

[MarcoRossignoli]

Guys we fixed the bug...can someone give it a try using nightly build tomorrow? https://github.com/tonerdo/coverlet/blob/master/Documentation/ConsumeNightlyBuild.md

Thank's a lot to all for the help!Really appreciated.

[zmhh]

Edit: It appears I tried to use the nightly before it was produced. Will check again for the updated package later.

[MarcoRossignoli]

Edit: It appears I tried to use the nightly before it was produced. Will check again for the updated package later.

Yep will be build at midnight...try tomorrow.

[zmhh]

Edit: Issue is resolved using nightly version 1.2.38-g34d6dc5ff6. Thanks Marco!

[MarcoRossignoli]

Glad to hear!