Fsharp: VS2017 15.8 P3, regression: Overload resolution on trivial types fails, existing projects don't compile anymore

[abelbraaksma]

With Visual Studio 2017 15.8 Preview 3, overload resolution fails to compile my existing projects that compiled fine in Preview 2 and earlier editions.

It seems limited to overload resolution that involved obj, but I'm not 100% sure of that.

Repro steps

I think I captured the issue with this repro, it appears to typically evolve around Option.ofObj in my existing code, but I cannot ascertain it is limited to that function:

```f#
module TestOfObj =

type [<AllowNullLiteral>] ITest1 = interface end

type Methods =
    | Test1 of ITest1
    | Other of obj

    static member ofObject t1 = Option.ofObj t1 |> Option.map Test1
    static member ofObject t0 = Option.ofObj t0 |> Option.map Other

    static member convert (x: obj) =
        match x with
        | :? ITest1 as one -> Methods.ofObject one
        | :? obj as one -> Methods.ofObject one

The error shows in FSI and in a normal project.

### Expected behavior

The above code should compile fine (possibly with a warning on `:? obj` always succeeding).

### Actual behavior

Compile errors are raised:

> error FS0041: A unique overload for method 'ofObject' could not be determined based on type information prior to this program point. A type annotation may be needed. Candidates: static member Methods.ofObject : t0:'a -> Methods option when 'a : null, static member Methods.ofObject : t1:'a -> Methods option when 'a :> ITest1 and 'a : null

### Known workarounds

~I don't yet know of a workaround, I'll check.~

Edit: even though the types are clearly inferred (mouseover shows that, as does the type definition when you compile it without the failing member), adding the types prevents the error:

```f#
static member ofObject (t0: obj) = Option.ofObj t0 |> Option.map Other
static member ofObject (t1: ITest1) = Option.ofObj t1 |> Option.map Test1

Though this may not always be possible if the members are in a compiled library, but I doubt the issue rises there. This seems to suggest it really is limited to inferred type overload resolution (if that's a thing).

Related information

Visual Studio 2017, 15.8 Preview 3. Tested with .NET Framework 4.6 and 4.7.x. Target F# runtime 4.4.3.0.

This is possibly a severe regression as it breaks existing projects to be compiled (unless of course this was a deliberate backwards incompatible design choice that I missed somehow).

[cartermp]

Can confirm that this code works on 15.7.4 as well, so this is definitely a regression. @dsyme any ideas?

[cartermp]

My wild guess is that this is related to the subsumption work, but that's just a guess

[TIHan]

I tested this with latest master and it seems to work properly.

[dsyme]

OK, it fails with dev15.8 (some work went direct into that branch and is not yet back in master). It's absolutely related to the 4.5 addition of subsumption on union constructors.

[TIHan]

Gotcha. I see now.

[cartermp]

Unless we get a quick fix in ASAP, we'll pull this feature. Definitely a small enough thing that we can ship without it, especially since it causes a regression. @abelbraaksma THANK YOU for the report! Saved our butts here.

[cartermp]

Okay, we'll pull this feature out of F# 4.5. @KevinRansom or @dsyme could we queue up the PR to remove #5030 and add a regression test with this?

module TestOfObj =

    type [<AllowNullLiteral>] ITest1 = interface end

    type Methods =
        | Test1 of ITest1
        | Other of obj

        static member ofObject t1 = Option.ofObj t1 |> Option.map Test1
        static member ofObject t0 = Option.ofObj t0 |> Option.map Other

        static member convert (x: obj) =
            match x with
            | :? ITest1 as one -> Methods.ofObject one
            | :? obj as one -> Methods.ofObject one

[cartermp]

Once pulled out, I'll also update the RFC repo

[abelbraaksma]

THANK YOU for the report! Saved our butts here.

Just glad to be of help :). Pity it means you have to pull a whole feature though... No easy/quick fix?

[cartermp]

Unfortunately not. The change affected overload resolution, which is very tricky.

[cartermp]

Fixed by #5261

[abelbraaksma]

Great that it's fixed! :)

[cartermp]

NP! And thanks for the repro - this is now officially a regression test: https://github.com/Microsoft/visualfsharp/pull/5261/files#diff-b0f8e0c99d39cd1cc1e7f2e310f8fbec