I'm trying to add F# to a code golf website. I recently added Java to that site and I'm in the process of adding C#.
I've been having trouble setting up a .NET Core 3 container with the F# compiler that can compile a trivial script within a couple seconds. The site has a seven second time limit for building/running scripts. For Java and C#, I can compile the script within about a second. For F#, it's taking 6 or more seconds to build the script with dotnet build --no-restore. I had a similar problem with C# was able to save a few seconds by running the compiler directly. I tried running the F# compiler directly and that only takes three seconds, but I'm having trouble running the created assembly. I also made a custom program that embeds the F# compiler and it takes about 6 seconds to compile and run the trivial script.
Hopefully I'm doing something wrong. My three attempts at dockerfiles are here. I'm not sure if opening an issue here is appropriate, but I wasn't sure where I could get suggestions for how to fix this.
Is there anything I can do to make this faster? The scripts will be fairly simple and won't need to use any features beyond the core F# library and a few core .NET libraries.
It seems like it should be possible to compile and run a simple program in significantly less than seven seconds, but it's been eluding me.
I'm pretty sure F# would be better for code golf than C# and Java.
Just a hunch, but couldn't you use the compiler services for both C# and F#? That way the compiler can stay in memory, since most time will be spent starting up.
For timings, compare how FSI.exe does it. A typical script takes a few ms to compile and execute. It is open source, you could check out how it operates.
I experience a similar slow startup times with FSI on Ubuntu 19.10 using the .NET 5 preview running this simple script.
#!/bin/sh
#if run_with_bin_sh
exec dotnet fsi --langversion:preview --exec $0 $*
#endif
#r "nuget: FSharp.Data"
printfn "hello world"
/usr/bin/time -v ./foo.fsx
hello world
Command being timed: "./foo.fsx"
User time (seconds): 4.42
System time (seconds): 0.41
Percent of CPU this job got: 127%
Elapsed (wall clock) time (h:mm:ss or m:ss): 0:03.79
Average shared text size (kbytes): 0
Average unshared data size (kbytes): 0
Average stack size (kbytes): 0
Average total size (kbytes): 0
Maximum resident set size (kbytes): 170384
Average resident set size (kbytes): 0
Major (requiring I/O) page faults: 17
Minor (reclaiming a frame) page faults: 67186
Voluntary context switches: 5053
Involuntary context switches: 290
Swaps: 0
File system inputs: 0
File system outputs: 7792
Socket messages sent: 0
Socket messages received: 0
Signals delivered: 0
Page size (bytes): 4096
Exit status: 0
@cartermp do you want a seperate issue for that or would the slow startup time of FSI be covered here too?
I also noticed that fsi took at least 3 seconds to startup and run a trivial script for .NET Core 3.1.201.
For Code Golf, running the compiler as a service is certainly an interesting idea. The service would likely need to run outside of the sandboxed environment that scripts normally run in. Then we could compile an assembly outside of the sandbox and run it inside of one. My understanding is that we wouldn't have to worry about security, beyond the normal concerns, because there's no way to write F# code that runs at compile time. Is this true?
Is there any documentation for running the compiler as a service that's compatible with running in a .NET Core 3 Linux environment?
I would also be very curious to know where all of the time is actually spent during startup. 3 - 7 seconds is a substantial amount of time.
It's absolutely possible to write F# code that runs at compile time, via a mechanism known as Type Providers. These typically have to be referenced via a nuget though, so would be white-listable in that way if you were worried about it.
As far as running the compiler service in a .net core 3 linux environment, the existing VS Code support for F# depends entirely on this feature set. The API docs and tutorials for the FSharp.Compiler.Service are available here, and are derived from the code in this repo which also contains samples.
@SirBogman Unfortunately, there's not a lot to go on here. If you can run dotnet msbuild /clp:PerformanceSummary on a clean (unbuilt) version if your project _before_ attempting any direct fsc invocations that would help understand where time is being spent when things reach MSBuild. Depending on what you're doing, you can see some wild results. For example, a simple console F# app:
11 ms Copy 3 calls
13 ms ResolvePackageFileConflicts 1 calls
13 ms WriteCodeFragment 1 calls
28 ms ProcessFrameworkReferences 1 calls
37 ms GenerateDepsFile 1 calls
43 ms GenerateRuntimeConfigurationFiles 1 calls
72 ms ResolvePackageAssets 1 calls
80 ms ResolveAssemblyReference 1 calls
1823 ms Fsc 1 calls
But here's a small solution with a C# project that only defines a single interface and an F# project that consumes the default interface member:
18 ms WriteCodeFragment 2 calls
31 ms ProcessFrameworkReferences 2 calls
42 ms GenerateRuntimeConfigurationFiles 1 calls
43 ms GetReferenceNearestTargetFrameworkTask 1 calls
53 ms GenerateDepsFile 2 calls
78 ms ResolvePackageAssets 2 calls
139 ms Csc 1 calls
157 ms ResolveAssemblyReference 2 calls
277 ms Copy 5 calls
1907 ms Fsc 1 calls
3526 ms MSBuild 6 calls
In the latter the F# compiler has to import another assembly (the C# one) but it's still proportional to the Hello World case since there are numerous assemblies to import for a basic .NET Core project. But you can see that there's MSBuild nonsense happening in the latter one that dominates everything else.
So the first step is to get a performance summary to see, once MSBuild is involved, where time is spent.
The .NET CLI also does have some overhead where it resolves the correct SDK to call into based on the project you have, I don't know how to profile that.
Once you've established how much time is spent in Fsc, you can find out exactly where time is being spent is via a trace, which you can collect with dotnet-track and analyze in a tool that can look at ETL traces. This is the only way to profile it reliably.
@realvictorprm That script is expected to be slow. FSI has always had slow startup times since it has to do a lot of things at initialization time to accept arbitrary code and compile/execute it quickly. Additionally, #r: nuget involves a NuGet resolution and various other MSBuild invocations, which is always expensive. Your timings don't seem out of the ordinary to execute that script cold.
Step 7/11 : RUN time /usr/bin/dotnet msbuild /clp:PerformanceSummary -p:Configuration=Release -restore:false /source/myapp1.fsproj
---> Running in a7f5d0815ffb
Microsoft (R) Build Engine version 16.5.0+d4cbfca49 for .NET Core
Copyright (C) Microsoft Corporation. All rights reserved.
myapp1 -> /source/bin/Release/netcoreapp3.1/myapp1.dll
Project Evaluation Performance Summary:
229 ms /source/myapp1.fsproj 1 calls
Project Performance Summary:
3276 ms /source/myapp1.fsproj 1 calls
Target Performance Summary:
0 ms PrepareResourceNames 1 calls
0 ms CreateCustomManifestResourceNames 1 calls
0 ms InitializeSourceControlInformation 1 calls
0 ms GetReferenceAssemblyPaths 1 calls
0 ms AfterResolveReferences 1 calls
0 ms CoreBuild 1 calls
0 ms BeforeCompile 1 calls
0 ms GenerateAssemblyInfo 1 calls
0 ms PrepareProjectReferences 1 calls
0 ms BeforeBuild 1 calls
0 ms ResGen 1 calls
0 ms CreateSatelliteAssemblies 1 calls
0 ms BeforeResGen 1 calls
0 ms ResolvePackageDependenciesForBuild 1 calls
0 ms SetWin32ManifestProperties 1 calls
0 ms GetFrameworkPaths 1 calls
0 ms BeforeResolveReferences 1 calls
0 ms Compile 1 calls
0 ms IncludeTransitiveProjectReferences 1 calls
0 ms GetTargetPath 1 calls
0 ms AfterResGen 1 calls
0 ms ResolveLockFileAnalyzers 1 calls
0 ms AfterCompile 1 calls
0 ms BuildOnlySettings 1 calls
0 ms AddSourceRevisionToInformationalVersion 1 calls
0 ms PrepareResources 1 calls
0 ms _DefaultMicrosoftNETPlatformLibrary 1 calls
0 ms ExpandSDKReferences 1 calls
0 ms _CheckForUnsupportedCppNETCoreVersion 1 calls
0 ms EnableIntermediateOutputPathMismatchWarning 1 calls
0 ms _CopySourceItemsToOutputDirectory 1 calls
0 ms CollectPackageReferences 1 calls
0 ms CollectFSharpDesignTimeTools 1 calls
0 ms ResolveReferences 1 calls
0 ms _PopulateCommonStateForGetCopyToOutputDirectoryItems 1 calls
0 ms _SplitProjectReferencesByFileExistence 1 calls
0 ms ResolveSDKReferences 1 calls
0 ms _GetAppHostPaths 1 calls
0 ms GetAssemblyAttributes 1 calls
0 ms PrepareForRun 1 calls
0 ms AfterBuild 1 calls
0 ms _CheckForObsoleteDotNetCliToolReferences 1 calls
0 ms _CheckForLanguageAndFeatureCombinationSupport 1 calls
0 ms _ComputeNETCoreBuildOutputFiles 1 calls
0 ms _CheckForUnsupportedNETCoreVersion 1 calls
0 ms _GetProjectJsonPath 1 calls
0 ms CoreResGen 1 calls
0 ms _ComputePackageReferencePublish 1 calls
0 ms Build 1 calls
0 ms _GenerateCompileInputs 1 calls
0 ms _CheckForUnsupportedAppHostUsage 1 calls
0 ms ComputePackageRoots 1 calls
0 ms GetAssemblyVersion 1 calls
1 ms _CheckForCompileOutputs 1 calls
1 ms ResolveLockFileCopyLocalFiles 1 calls
1 ms _GetFrameworkAssemblyReferences 1 calls
1 ms GetTargetPathWithTargetPlatformMoniker 1 calls
1 ms _GenerateSatelliteAssemblyInputs 1 calls
1 ms SplitResourcesByCulture 1 calls
1 ms _GenerateRuntimeConfigurationFilesInputCache 1 calls
1 ms _GetCopyToOutputDirectoryItemsFromTransitiveProjectReferences 1 calls
1 ms _GetCopyToOutputDirectoryItemsFromThisProject 1 calls
1 ms AssignTargetPaths 1 calls
1 ms ResolveLockFileReferences 1 calls
2 ms CheckForDuplicateItems 1 calls
2 ms _CheckForInvalidConfigurationAndPlatform 1 calls
2 ms _ComputeUserRuntimeAssemblies 1 calls
2 ms _ComputeReferenceAssemblies 1 calls
2 ms GenerateTargetFrameworkMonikerAttribute 1 calls
2 ms _GetProjectReferenceTargetFrameworkProperties 1 calls
2 ms CreateGeneratedAssemblyInfoInputsCacheFile 1 calls
3 ms PrepareForBuild 1 calls
3 ms CopyFilesToOutputDirectory 1 calls
3 ms ResolveFrameworkReferences 1 calls
3 ms IncrementalClean 1 calls
3 ms CheckForImplicitPackageReferenceOverrides 1 calls
4 ms _SetEmbeddedWin32ManifestProperties 1 calls
5 ms UpdateAspNetToFrameworkReference 1 calls
5 ms ApplyImplicitVersions 1 calls
7 ms _HandlePackageFileConflicts 1 calls
7 ms _CleanGetCurrentAndPriorFileWrites 1 calls
9 ms ResolveTargetingPackAssets 1 calls
9 ms _CopyOutOfDateSourceItemsToOutputDirectory 1 calls
10 ms GetCopyToOutputDirectoryItems 1 calls
10 ms _CreateAppHost 1 calls
11 ms RedirectTPReferenceToNewRedistributableLocation 1 calls
11 ms _CollectTargetFrameworkForTelemetry 1 calls
14 ms _GetRestoreProjectStyle 1 calls
14 ms GenerateFSharpTextResources 1 calls
16 ms CoreGenerateAssemblyInfo 1 calls
19 ms FindReferenceAssembliesForReferences 1 calls
27 ms _CopyFilesMarkedCopyLocal 1 calls
50 ms GenerateBuildDependencyFile 1 calls
53 ms RedirectFSharpCoreReferenceToNewRedistributableLocation 1 calls
58 ms GenerateBuildRuntimeConfigurationFiles 1 calls
59 ms ResolveProjectReferences 1 calls
73 ms ProcessFrameworkReferences 1 calls
76 ms ResolveAssemblyReferences 1 calls
92 ms ResolvePackageAssets 1 calls
2573 ms CoreCompile 1 calls
Task Performance Summary:
0 ms GetAssemblyVersion 1 calls
0 ms Delete 1 calls
0 ms AssignCulture 1 calls
0 ms FindAppConfigFile 1 calls
0 ms ReadLinesFromFile 1 calls
0 ms ResolveFrameworkReferences 1 calls
1 ms CheckForDuplicateFrameworkReferences 1 calls
1 ms ConvertToAbsolutePath 1 calls
1 ms Message 3 calls
1 ms JoinItems 1 calls
1 ms RemoveDuplicates 2 calls
1 ms MSBuild 1 calls
1 ms AssignTargetPath 6 calls
1 ms Touch 1 calls
1 ms CheckForDuplicateItems 3 calls
1 ms ApplyImplicitVersions 1 calls
1 ms GetPackageDirectory 6 calls
1 ms CheckIfPackageReferenceShouldBeFrameworkReference 2 calls
2 ms FSharpEmbedResourceText 1 calls
2 ms Hash 2 calls
2 ms CheckForImplicitPackageReferenceOverrides 1 calls
2 ms WriteLinesToFile 4 calls
2 ms FSharpEmbedResXSource 1 calls
3 ms FindUnderPath 5 calls
3 ms Telemetry 1 calls
4 ms GetFrameworkPath 1 calls
5 ms ResolvePackageFileConflicts 1 calls
6 ms MakeDir 2 calls
8 ms ResolveTargetingPackAssets 1 calls
8 ms GetRestoreProjectStyleTask 1 calls
8 ms CallTarget 2 calls
9 ms CreateAppHost 1 calls
15 ms WriteCodeFragment 1 calls
22 ms ResolveAppHosts 1 calls
29 ms Copy 4 calls
48 ms GenerateDepsFile 1 calls
48 ms ProcessFrameworkReferences 1 calls
57 ms GenerateRuntimeConfigurationFiles 1 calls
75 ms ResolveAssemblyReference 1 calls
91 ms ResolvePackageAssets 1 calls
2571 ms Fsc 1 calls
real 0m 3.98s
user 0m 3.82s
sys 0m 0.46s
It might also be useful to point out that much of the work being done above is unnecessary if your dependency graph didn't change between compiles. In that case, all you have to do is compile, emit a single dll, and run that.
This is how Try .NET is as responsive as it is. The example here shows timings which include round-trips to the compilation service, i.e. most of the time is spent in network I/O. (The example is C# because we don't have F# WASM support in place, but the same strategy should apply regardless.)

Try .NET is pretty cool.
I tried compiling with fsc directly, but I have trouble running the resulting program and don't understand why.
time /usr/bin/dotnet exec \
/usr/share/dotnet/sdk/3.1.201/FSharp/fsc.exe \
--nologo \
--optimize \
--targetprofile:netcore \
--target:exe \
--noframework \
-r:/root/.nuget/packages/fsharp.core/4.7.0/lib/netstandard2.0/FSharp.Core.dll \
-r:/usr/share/dotnet/shared/Microsoft.NETCore.App/3.1.3/System.Collections.dll \
-r:/usr/share/dotnet/shared/Microsoft.NETCore.App/3.1.3/System.Private.CoreLib.dll \
-r:/usr/share/dotnet/shared/Microsoft.NETCore.App/3.1.3/System.Runtime.Extensions.dll \
-r:/usr/share/dotnet/shared/Microsoft.NETCore.App/3.1.3/System.Runtime.Numerics.dll \
-r:/usr/share/dotnet/shared/Microsoft.NETCore.App/3.1.3/System.Runtime.dll \
-r:/usr/share/dotnet/shared/Microsoft.NETCore.App/3.1.3/System.Net.Requests.dll \
-r:/usr/share/dotnet/shared/Microsoft.NETCore.App/3.1.3/System.Net.WebClient.dll \
-r:/usr/share/dotnet/shared/Microsoft.NETCore.App/3.1.3/netstandard.dll \
-o /source/Program.exe \
/source/Program.fs
This seems to be the minimum set of references needed to build a trivial program. If I leave them out, I get this:
error FS0193: Could not load file or assembly 'Microsoft.Build.Tasks.v4.0, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a'. The system cannot find the file specified.
When I try to run the resulting assembly, I get this:
Unhandled exception. System.MethodAccessException: Entry point must have a return type of void, integer, or unsigned integer.
Command terminated by signal 6
Here's the code:
[<EntryPoint>]
let main argv =
printfn "Hello World from F#!"
argv |> Seq.iteri (printfn "Arg %i: %s")
0
When I load the assembly in a custom program and try to run it by reflection, I get this:
Unhandled exception. System.Reflection.TargetInvocationException: Exception has been thrown by the target of an invocation.
---> System.MissingMethodException: Method not found: 'Void Microsoft.FSharp.Core.PrintfFormat`5..ctor(System.String)'.
--- End of inner exception stack trace ---
at System.RuntimeMethodHandle.InvokeMethod(Object target, Object[] arguments, Signature sig, Boolean constructor, Boolean wrapExceptions)
at System.Reflection.RuntimeMethodInfo.Invoke(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)
at System.Reflection.MethodBase.Invoke(Object obj, Object[] parameters)
at Runner.runCode(Assembly assembly, String[] args) in /runner_source/Runner.fs:line 13
at Runner.main(String[] args) in /runner_source/Runner.fs:line 23
I'm not sure how to address either of these issues.
Thanks for all of the suggestions. I was able to get this to work well enough for my use case.
After discussing with the maintainer of the code golf site, we didn't think it was worth the extra complexity to have a compiler service running outside of the sandbox. None of the other languages need that.
So I looked into trying to make it a bit faster. I wrote a small application that wrapped the FSharp.Compiler.Service and compiles and runs a script.
For a trivial script, it initially took 4 seconds, which is too long for the 7 second timeout.
By publishing with ReadyToRun, the time went down from 4 to 2.75 seconds.
At this point I noticed that after my app compiled the script, every .NET Core SDK assembly was loaded in the AppDomain.
I thought this might be contributing to the compile time.
After enabling trimming, the time went down from 2.75 to 2.5 seconds.
I noticed there were still a large number of assemblies I felt were unnecessary for code golf and manually removed them. That reduced the time fro 2.5 to 2.15 seconds. At this point, the compile time is acceptable for my use case, although it's not ideal.
There were some assemblies that I wanted to remove, but could not because I think they were needed by FSharpCore: System.Net.Requests, and System.Net.WebClient.
Here's the set of assemblies I ended up with, after removing everything that didn't cause an error and that I didn't feel would be useful for code golf.
FSharp.Compiler.Service.dll
FSharp.Core.dll
System.Collections.Concurrent.dll
System.Collections.Immutable.dll
System.Collections.NonGeneric.dll
System.Collections.dll
System.Console.dll
System.Diagnostics.TraceSource.dll
System.IO.FileSystem.dll
System.IO.IsolatedStorage.dll
System.IO.MemoryMappedFiles.dll
System.Linq.dll
System.Net.Requests.dll
System.Net.WebClient.dll
System.Private.CoreLib.dll
System.Reflection.Metadata.dll
System.Runtime.Extensions.dll
System.Runtime.InteropServices.RuntimeInformation.dll
System.Runtime.InteropServices.dll
System.Runtime.Numerics.dll
System.Runtime.dll
System.Text.RegularExpressions.dll
System.Threading.Thread.dll
netstandard.dll
The docker image is just over 100 MB.
This issue can probably be closed now, although I would still appreciate any speed improvements in the compiler startup process. Perhaps less time could be spent loading assemblies that aren't strictly required?
You could try to ILMerge the assemblies, so that it only has to load a single big one instead of 24 small ones.
You could try to ILMerge the assemblies, so that it only has to load a single big one instead of 24 small ones.
That's an interesting idea. I tried using PublishSingleFile, which I guess isn't exactly the same thing. I couldn't get it to work. I had problems referencing assemblies. I couldn't reference my single file application when invoking the compiler, because it's not a valid Win32 PE file (I'm using Linux). When I tried to include a few assemblies as separate dlls, the compiler API couldn't load them.
Can IL merge be used for .NET Core 3.1.200?
F# is now live on code-golf.io!
It has slightly slower startup time than any of the other languages, taking about 2 seconds for compiler startup.
If anyone is interested, the code is here for setting up the application containing the F# compiler, trimming it, and creating a container.
Thanks @SirBogman - bit of a shame that startup time before anything ever reaches fsc inhibits the use of F# here.
F# is a great language. It's definitely not the best for code golf, but it's still fun, as long as you don't do it in production code.
I did capture one performance trace with the method you described. I haven't figured out how to look at it yet though :)
Typically the best tool for looking at the trace is with PerfView on Windows. I'm not sure if there are Linux equivalents.
I'll try that. I had tried Windows Performance Analyzer and I couldn't get it to open the file.
@SirBogman if you're on linux you can use speedscope to view them, as described here
Thanks, I was able to get PerfView to open the file. It looks like a lot of time was spent in JIT but I'm not sure if I was using ReadyToRun at the time. I'll try to capture a new file and see if there's anything interesting in it.
That's an interesting idea. I tried using PublishSingleFile, which I guess isn't exactly the same thing. I couldn't get it to work. I had problems referencing assemblies. I couldn't reference my single file application when invoking the compiler, because it's not a valid Win32 PE file (I'm using Linux). When I tried to include a few assemblies as separate dlls, the compiler API couldn't load them.
Yeah, PublishSingleFile packages all the small files into a zip file, then extracts it on startup into temp and runs it normally.
Can IL merge be used for .NET Core 3.1.200?
I used https://github.com/gluck/il-repack the last time.
It should work for all pure IL assemblies (not native code), but it would need to be executed before Ready2Run.
So the pipeline should be compile => Link => Merge => Ready2Run.
Of the two second run time to start the wrapper program, compile a trivial program, and run it, a substantial amount of time appears to be spent in JIT, even though the assemblies were all ReadyToRun (except the dynamic golf1 assembly).

Does anyone have thoughts on why so much time is spent in JIT with ready to run? Could it be related to generic type instantiations that are created at runtime?
It appears to save about 100 msec by disabling Tiered Compilation.

When I disabled ReadyToRun, it took about 7 seconds to run the program, as I originally reported. It appears that JIT accounts for the added time.

Closing out old discussion
It appears that JIT accounts for the added time.
Not sure it was suggested here before, but can you ngen your binaries? That should eliminate most of the JIT time.
Does anyone have thoughts on why so much time is spent in JIT with ready to run? Could it be related to generic type instantiations that are created at runtime?
Prejitting of methods with tail calls is not yet supported by Ready To Run, see dotnet/runtime#5857. That's the most likely explanation. cc @jkotas.
If you look further down in the jit time report it shows all the methods that are jitted. Perhaps that will show some other patterns.
@AndyAyersMS Thanks, that does seem like the most likely explanation. Hopefully it will be implemented eventually.