Fable: Question on Fable.Repl

[nojaf]

I'd like to know something regarding the Fable REPL.
How is the compilation being done differently than with normal Fable?

I can imagine the FCS is not being used as lexer for the source code? If this is the case, are the same AST constructs used as the FCS?

I'm asking for the ast-viewer, in orde to have a good visualization all AST info needs to be there. So I'm wondering if the REPL currently pulls this off?

[alfonsogarciacaro]

AFAIK the AST produced by FCS in the REPL is exactly the same as the one produced by "normal" FCS. The lexer is there and even the assembly reader. The only difference is the REPL uses the fable branch of @ncave FCS fork instead of fsharp/FSharp.Compiler.Service source but, if I'm not mistaken the only differences are a few polyfills for code that doesn't work in JS as such and removing the part about IL generation. You can check the PR for compiling FCS to JS here: https://github.com/fsharp/FSharp.Compiler.Service/pull/688

[nojaf]

So @ncave is able to compile ast.fs with Fable?

That is great news. Any ideas what it would take to compile a SynExpr in Fable? Would it suffice to include some files from the ncave's fork in my project (using paket github)?

[alfonsogarciacaro]

This is the project we use to generate the REPL. I guess you don't need the Fable part so probably just referencing @ncave fork is enough (remember to use fable branch and run the fcs/build CodeGen.Fable target in advance). With that, you should be able to use FCS API normally as in .NET. When you have the project set up, let me know and I can help you with the JS compilation. For reference, this is the build target we use to compile FCS+Fable to JS.

BTW, we use REPL to mean two things, so it can be a bit confusing. The REPL as generated in this repository is the JS bundle that contains the FCS+Fable source. The UI for the REPL web is in the fable-compiler/repl2 repository atm.

[nojaf]

Hi @alfonsogarciacaro, I've followed your instructions and I got it to work 👌!
Next challenge is to serialize the FCS AST on the server to the client.
Current setup looks like standard saturn so not sure who or what is creating the json.

I believe standard JSON.NET atm, which cannot deal with self references:
"Self referencing loop detected for property 'StartRange' with type 'Microsoft.FSharp.Compiler.Range+range'. Path 'Ok[0].Let[1][0].Binding[7].Named[0].Wild.StartRange'."

[alfonsogarciacaro]

Ahá, if you manage to serialize the F# AST, you tell me ;) All my attempts haven't given any result so far (didn't try too hard though). Question, why do you generate the AST in the server? I thought your purpose was to do the parsing directly on the client side as the Fable REPL does.

[nojaf]

Ah no, the grand master plan is to have a nice visualization of the AST on the client.
The ast-viewer now just shows a ToString of the AST.

So on the Client I'm able to compile things like

let constant = Ast.Const(Ast.SynConst.Unit, Range.range.Zero)

Which I create on the server with the _real_ FCS. Plan was to serialize that, send to client and create a whimsical UI.

I guess I could try getting the AST on the client but I'm having mixed feelings of that outcome. There is no guarantee that the ncave fable FCS produces the same AST as the fsc would. Or am I wrong to have this concern?

[alfonsogarciacaro]

The FCS from ncave's fork compiled by Fable produces exactly the same AST as FCS on dotnet. Fable itself uses the AST for JS generation and we didn't have to change anything for the REPL version. All the editor features you see in the Fable REPL (tooltips, autocompletion, go to definition, error highlighting) come from FCS exactly the same way as they work for FSAC or Ionide, for example.

[alfonsogarciacaro]

That said, if you prefer to do the parsing on the server (it's true the FCS compiled to JS is a bit heavy to load in the browser and it also requires many assemblies) and you want to have nice visualization, I would create a custom AST that works with your visualization tools. If you use Fable on the frontend, you can share the types for your AST and do the JSON serialization with Thoth.Json (or use Fable.Remoting).

[ncave]

@nojaf There is also the F# AST printer that you can perhaps customize to your requirements.

[nojaf]

Thanks all for the information. My question is hereby answered.

[jindraivanek]

I did quick experiment with serializing of AST and with JsonSerializerSettings(..., PreserveReferencesHandling = PreserveReferencesHandling.Objects, ReferenceLoopHandling = ReferenceLoopHandling.Ignore) it almost work.

   (false,
    [Binding
       (None,NormalBinding,false,false,[],
        PreXmlDoc ((1,5),Microsoft.FSharp.Compiler.Ast+XmlDocCollector),
        SynValData (None,SynValInfo ([],SynArgInfo ([],false,None)),None),
        Named
          (Wild tmp.fsx (1,4--1,5) IsSynthetic=false,x,false,None,
           tmp.fsx (1,4--1,5) IsSynthetic=false),None,
        Const (Int32 1,tmp.fsx (1,8--1,9) IsSynthetic=false),
        tmp.fsx (1,4--1,5) IsSynthetic=false,
        SequencePointAtBinding tmp.fsx (1,0--1,9) IsSynthetic=false)],
    tmp.fsx (1,0--1,9) IsSynthetic=false)

after serialize-deserialize

   (false,
    [Binding
       (None,NormalBinding,false,false,[],
        PreXmlDoc ((0,0),Microsoft.FSharp.Compiler.Ast+XmlDocCollector),
        SynValData (None,SynValInfo ([],SynArgInfo ([],false,None)),None),
        Named
          (Wild unknown (0,0--0,0) IsSynthetic=false,,false,None,
           unknown (0,0--0,0) IsSynthetic=false),None,
        Const (Int32 1,unknown (0,0--0,0) IsSynthetic=false),
        unknown (0,0--0,0) IsSynthetic=false,
        SequencePointAtBinding unknown (0,0--0,0) IsSynthetic=false)],
    unknown (0,0--0,0) IsSynthetic=false)

@alfonsogarciacaro it is serializing AST important feature for something? It is worth pursuing?

[alfonsogarciacaro]

Good one @jindraivanek! My main use case for serializing the AST in Fable would be precompiled libraries as we have for the REPL. Currently the main limitation is we cannot inline functions from the library because we need the AST for that and the .dll is not enough. If we could save the AST for the inline functions and load it later (for the REPL it needs to be done from JS, so no Newtonsoft.Json) we could finally inline the functions without parsing the whole library every time. As far as I can see, the only thing that is missing is the range information. This shouldn't be an obstacle for the inline functions :+1: