Fsharp: Module-scoped records and DUs are null in tests

[TysonMN]

Succinct description of the issue

Records and discriminated unions defined at the module level are null in tests. If defined with [<Struct>], then they take their default value.

Repro steps

Here is a ZIP that demonstrates the problem. Here is the F# code in question.

```F#
module Tests

open Xunit

type MyDiscriminatedUnion = MyDiscriminatedUnion of unit

[] // passes
let testScopedDiscriminatedUnion is not null() =
let testScopedDiscriminatedUnion = MyDiscriminatedUnion ()
obj.ReferenceEquals(testScopedDiscriminatedUnion, null) |> Assert.False

let moduleLevelDiscriminatedUnion = MyDiscriminatedUnion ()
[] // fails
let moduleLevelDiscriminatedUnion is not null() =
obj.ReferenceEquals(moduleLevelDiscriminatedUnion, null) |> Assert.False

type MyRecord = { Field: Unit }

[] // passes
let testScopedRecord is not null() =
let testScopedRecord = { Field = () }
obj.ReferenceEquals(testScopedRecord, null) |> Assert.False

let moduleScopedRecord = { Field = () }
[] // fails
let moduleScopedRecord is not null() =
obj.ReferenceEquals(moduleScopedRecord, null) |> Assert.False
```

Expected behavior

All tests pass.

Actual behavior

Two tests fails as indicated by the comments.

Known workarounds

The main workaround is given in the above tests that pass, which is to define the record and discriminated union in the test.

Another workaround that I don't understand is exhibited in the state of the fsharp-hedgehog repository at this commit. In that commit, the NuGet package dependencies are specified and obtained via paket. See this test, which passes at that commit. In the process of switching from using paket to <PackageReference /> entries (c.f. PR https://github.com/hedgehogqa/fsharp-hedgehog/pull/207), we encountered the present issue (c.f. issue https://github.com/hedgehogqa/fsharp-hedgehog/issues/201).

Related information

Provide any related information (optional):

• Operating system: WIndows 10 Enterprise Version 10.0.18362 Build 18362
• .NET Runtime kind: .NET Core
• Editing Tools: Visual Studio Enterprise 2019 Version 16.6.3

[abelbraaksma]

Weird, it's as as if the normal module init code is not run in these tests.

[cartermp]

This isn't an F# issue - it's just a fundamental mismatch between how the xUnit test runner works and how you can define values outside the scope of test methods F#. In an F# program where you define these as functions and call them, the module-declared values are initialized. But xUnit doesn't run like a normal program, made apparent here when running tests. You'll note that if you use NUnit, this doesn't occur, nor does it occur with an F# test framework like Expecto where tests are functions/values executed as a part of normal program flow.

[TysonMN]

Ah, your suggestion to also attempt to reproduce this problem with another testing framework is an exceedingly good idea. I wish I had thought of that.

Another workaround that I don't understand is exhibited in the state of the fsharp-hedgehog repository at this commit. In that commit, the NuGet package dependencies are specified and obtained via paket. See this test, which passes at that commit. In the process of switching from using paket to <PackageReference /> entries (c.f. PR https://github.com/hedgehogqa/fsharp-hedgehog/pull/207), we encountered the present issue (c.f. issue https://github.com/hedgehogqa/fsharp-hedgehog/issues/201).

In the process of attempting to reproduce this problem with NUnit, I now understand that workaround.

The paket lock file was installing version 15.9.0 of Microsoft.NET.Test.SDK. My solution ZIP above did not explicitly mention Microsoft.NET.Test.SDK. It did explicitly mention xunit.runner.visualstudio version 2.4.1, which depends on version >= 15.0.0 of Microsoft.NET.Test.SDK. Without paket, I think Microsoft.NET.Test.SDK was being resolved to this minimum version 15.0.0.

I tested several versions of Microsoft.NET.Test.SDK. The key finding is that

• the two tests failed when using version <= 15.7.0 of Microsoft.NET.Test.SDK and
• all the tests passed when using version >= 15.8.0 of Microsoft.NET.Test.SDK.

Thank you very much for your comments @cartermp. Even though this wasn't an F# issue, you helped understand it much better :D