Fsharp: F# compiler treats FSharpFunc as sealed in latest update to F# 4.1

[krauthaufen]

Hi everyone,

I know it's bad practice to inherit FSharpFunc directly in user code but in certain scenarios i had to do that.
typeof int>.IsSealed reports false obviously but the new (4.1) compiler complains about FSharpFunc being sealed.

I understand the general idea of making all the concrete implementations sealed but why did you decide to treat the base-type as if it was sealed?

Would it be possible to make the base-type non-sealed again, since it just gives the system more flexibility. Couldn't it just be a warning of some sort? (e.g. inheriting FSharpFunc may cause your code to be broken in future versions of F#)

P.S.: the types in OptimiziedClosures (FSharpFunc<_,_,_>, etc.) are not sealed btw.

cheers

[vasily-kirichenko]

https://github.com/Microsoft/visualfsharp/issues/3020

[krauthaufen]

Hi, I'm aware that the implementers shall be sealed but why does the compiler treat the base type itself as if it was sealed?

The point here is that FSharpFunc itself is not sealed but the compiler forbids inheriting it with "cannot inherit a sealed type"

[eiriktsarpalis]

+1 I just came across this, and it's a breaking change in cases where defining stable closure names is a requirement, e.g. https://github.com/mbraceproject/MBrace.Core/blob/f7a2b8bc13ddd0f59928b82054f05a7be52e3755/src/MBrace.Core/Continuation/Builders.fs#L91

[dsyme]

@eiriktsarpalis I see this comment in your code

    // F# won't let use inherit OptimizedClosures.FSharpFunc<_,_,_> because of the unit return type
    inherit FSharpFunc<ExecutionContext, Continuation<'T> -> unit> ()

Is there a way to use a non-unit return type here? e.g. can you use a type parameter which you then instantiate to unit?

[eiriktsarpalis]

@dsyme I think we'd have to use an async style FakeUnit return type for the continuations.

[dsyme]

Hmm so https://github.com/Microsoft/visualfsharp/pull/3031/ (which implemented #3020) is not the change responsible here, since that is about making the closures that implement FSharpFunc sealed.

The change happened after F# 4.1 RTM. I think the change responsible is https://github.com/Microsoft/visualfsharp/pull/3283, which decodes FSharpFunc to the compiler's view on function types. There is an explicit opt-out when compiling FSharp.Core here https://github.com/Microsoft/visualfsharp/pull/3283/files#diff-5b9ab9dd9d7133aaf23add1048742031R4893.

@cartermp This is the second break because of that change - sorry - mea culpa, the other is https://github.com/Microsoft/visualfsharp/pull/3729

[cartermp]

Just as a note, we're likely past the window of getting this into a compiler update for 15.5, but for a 15.6 update we could conceivably take this. No timeline is in place for either release at the moment, but 15.5 will be sooner since we're now in Escrow.

[dsyme]

@cartermp I don't think this is super-urgent - 15.6 will be fine

I'm more concerned by https://github.com/Microsoft/visualfsharp/issues/3743.

[kzu]

Hi there, I'm on 15.6 Preview 4.0 and this fix isn't there yet. Any ETA you can share?

[manofstick]

@kzu

Here is a hack around this issue.

Although obviously not ideal, we have used this to updated the package.

[manofstick]

...and for any interested in the previously mentioned hack, @eiriktsarpalis has made this a cleaner solution by by statically linking the fix into the required fsharp project...

Obviously the proper solution would be nice, but this is quite clean now for an interim solution.

[praeclarum]

@Just to +1, Xamarin Live Player and my IDE make use of FSharpFunc overrides and this breaks F# support.

[zpodlovics]

Would it be possible to have both option available? I mean the FSharpFunc that is sealed and an another that is not sealed? Why this is important? Because a virtual call usually means at least four or five times slower calls.

A really unscientific and dumb call overhead test are already available for F# to test the direct vs virtual call performance difference:

https://gist.github.com/zpodlovics/73deb8b7a06d7d5ec374

Method call overhead:
No function                             1,462.0 MOps/s,   0.342 s
F# function                             2,242.2 MOps/s,   0.223 s
F# inline function                      2,057.6 MOps/s,   0.243 s
System.Func<int,int> function             231.9 MOps/s,   2.156 s
Delegate function                         230.9 MOps/s,   2.165 s
ClassA Static                           2,164.5 MOps/s,   0.231 s
ClassA plain                            2,008.0 MOps/s,   0.249 s
ClassA Interface call base plain          494.6 MOps/s,   1.011 s
ClassA Interface call base abstract       273.8 MOps/s,   1.826 s
ClassA Interface local impl               464.7 MOps/s,   1.076 s
ClassB Static                           2,183.4 MOps/s,   0.229 s
ClassB plain                            1,369.9 MOps/s,   0.365 s
ClassB overridden                         447.6 MOps/s,   1.117 s
ClassB ClassA cast plain                2,155.2 MOps/s,   0.232 s
ClassB ClassA cast overridden             478.9 MOps/s,   1.044 s
ClassB Interface plain                    392.2 MOps/s,   1.275 s
ClassB Interface overridden               268.2 MOps/s,   1.864 s
ClassB Interface local                    447.2 MOps/s,   1.118 s
StructC Static                          1,945.5 MOps/s,   0.257 s
StructC plain                           2,145.9 MOps/s,   0.233 s
StructC Interface call base plain          99.7 MOps/s,   5.016 s
StructC Interface local                    93.8 MOps/s,   5.328 s

Environment:
Mono 5.10.1.47
Ubuntu 16.04
F#: 4.1.33-0xamarin4+ubuntu1604b1
CPU: AMD Kaveri A10-7850K

CoreCLR started an effort to devirtualize lots of the calls, but that's not always possible. However sealed classes could make a huge difference in CoreCLR to, as it could be explicitly devirtualized by the JIT:

https://github.com/dotnet/coreclr/issues/1166
https://github.com/dotnet/coreclr/blob/master/Documentation/botr/virtual-stub-dispatch.md

[manofstick]

@praeclarum - can the hack be used to get you through? @eiriktsarpalis's use of static linking - as commented upon here - makes this quite seemless.

@zpodlovics - this is really a compiler issues as to how it views FSharpFunc - it isn't actually "sealed" as far as .net is concerned - if it was then we wouldn't actually derive proper implementations of functions! I have no idea the internals of what is going on within the compiler (I haven't looked, and don't have the time) but I assume it is some non-trivial dependency on some other functionality that has led to this situation.

[zpodlovics]

@manofstick The primitive abstraction is not sealed here https://github.com/Microsoft/visualfsharp/blob/master/src/fsharp/FSharp.Core/prim-types.fs#L2797 but the generated code (IL) will be sealed as of https://github.com/Microsoft/visualfsharp/pull/3031/commits/80c6f8b80739cbe128d2b8e4e4b743bdd9527a14

[manofstick]

@zpodlovics

Yes, which is fine. The issue with #3737 is that the FSharp compiler won't let you make your own types which derive directly from FSharpFunc - which is used by things such as the Quotations.Evaluator [hence the hack, which just uses c# to provide a dummy type around it] (i.e. internally when it creates lambda functions it will now create them sealed which is good.)

[praeclarum]

@manofstick No, the hack cannot be used. I need real FSharpFunc objects. Even with Don's help I still don't have a solution. Currently, Xamarin Live and my IDE are broke.

I get that you all want to make stuff fast, but you can't break APIs to do it. This should have been done in 5.0 not 4.X.

[manofstick]

@praeclarum

Hey I agree things shouldn't be broken in point releases (i.e. public interface which was only ever kinda public), but from my position way down in the food chain of contributors, it appears that the f# team is spread thinner than butter so I can see that errors can occur... (And the devirtualization thing, I'm guessing, will only really be helpful in some extremely edge cases, so I doubt we'd see much speed improvement - especially given that the f# compiler, where it can, will already create static functions internally where it can...)

Anyway, can you give me a few minutes of your time to explain why you are deriving from some lambda function?

i.e. if we have say

let TrivialLambdaFunction = fun x -> x * 2

which becomes (kinda)

[<Sealed>]
type TrivialLambdaFunction() =
    inherit Microsoft.FSharp.Core.FSharpFunc<int, int>()
    member __.Invoke x = x * x

Which no longer compiles in the latest (for last 6 to 12 months!) builds because F# says FSharpFunc is a sealed class, but that's neither here nor there - because you have stated that the hack doesn't help you (which it would if this was your only problem).

Why are you then deriving from TrivialLambdaFunction (well its kin anyway)?

[dsyme]

I've submitted a fix for this here https://github.com/Microsoft/visualfsharp/pull/4804

I've also submitted a fix to @praeclarum 's private NInterpret repository to show how to use the 4.1.23 FSharp.Compiler.Tools to unblock this until we get an updated compiler through the works.

[dsyme]

As context: This breaking change was my mistake, but I had been reluctant to fix it.

In the F# type system, the function type "int -> int" is a sealed type - there are no nominal subtypes of this type. It's always been like this. The breaking change in F# 4.1 was also to apply this rule to FSharpFunc<int,int> (the compiled form of int -> int) . The change happened because as of F# 4.1 we now regard these types as equivalent more or less all the time, with the same logical rules applying to each. That fixed some bugs, but broke things, unintentionally

I was way, way too slow to fix this, sorry. I kind of just kept closing my eyes/ears "la la la la" to the need for "exotic" implementors to provide their own nominal closure types with strong names for serialization and/or interpreter purposes. Finally I see we just need to support this.

[manofstick]

So from the sounds of the fix, if the hack had been applied then @praeclarum could have got this fixed earlier :-) But good that there is a proper fix coming down the pipeline. Thanks @dsyme.

[dsyme]

@manofstick Possibly but at considerable pain, having looked at his code.