Fable support in dotnet sdk

[alfonsogarciacaro]

@enricosada

I'm opening this issue to follow the discussion here. It's interesting to keep a conversation using git commits but probably this is a better medium ;)

I'm trying to summarize the three main parts of the integration and the tasks we need to accomplish:

Project initialization

User just types dotnet new -t fable as for other dotnet projects. More templates for different Fable projects (React, etc) will be available in the future.

TODO: Will you create the template @enricosada?

Package restore

We'll try to make things easier for the user, but not replace/hide npm completely. There should be a command like dotnet fable install fable-powerpack that will install a package both in package.json and the .fsproj (with a <FableReference Include="Fable.Core" /> item).

TODO: Decide if the user should type fable-core (the npm package name) or Fable.Core (the F# project name).

TODO: Add the references to package.json and the .fsproj. I can add this capability to Fable but, @enricosada, do you know what's the best way to edit the .fsproj XML?

TODO: The <FableReference Include="Fable.Core" /> must be translated to a ./node_modules/fable-core/Fable.Core.dll (the tool should be also able to detect if package.json is in a parent dir). I guess @enricosada can do the MSBuild trickery, but I can write a script for path resolution if needed.

Building

User will type dotnet fable (or dotnet fable build, or dotnet build fable) and the project will be compiled normally.

TODO: MSBuild should generate a response file with the F# compiler arguments (@enricosada? :wink:) and then call Fable with a reference to this file.

---

Other considerations

Right now it looks like Fable.Sdk and fable-compiler will be two different things. Fable.Sdk will only contain the Fable.targets, while fable-compiler remains an npm package. The main problem with this is it may be difficult to keep the versions in sync, so Fable.Sdk must pass the version to the compiler and warn the user if they differ. Probably Fable.Sdk should contain also the dotnet sdk tool that would redirect the dotnet fable calls to the locally installed fable.

[enricosada]

TODO: Will you create the template @enricosada?

Yes, waiting for preview5, atm in preview4 the dotnet-new is not extensibile, but will (ref https://github.com/dotnet/netcorecli-fsc/issues/31 ). if fable is ready before preview5, we can just use an clitool.

TODO: Add the references to package.json and the .fsproj. I can add this capability to Fable but, @enricosada, do you know what's the best way to edit the .fsproj XML?

the dotnet core sdk will support commands to modify the msbuild project programmatically (like forge), for example to add ref is something like dotnet ref add fable-powerpack 1.0.0 ( ref https://github.com/dotnet/cli/issues/4521 ). we can just call that command from dotnet fable install fable-powerpack after package.json is modified.

TODO: The must be translated to a ./node_modules/fable-core/Fable.Core.dll (the tool should be also able to detect if package.json is in a parent dir). I guess @enricosada can do the MSBuild trickery, but I can write a script for path resolution if needed.

The first part (FableReference -> msbuild Reference) already works in the POC, the recursive search is not difficult to add (it's just search and save directory in a property, used as base for fable references).

TODO: MSBuild should generate a response file with the F# compiler arguments (@enricosada? 😉) and then call Fable with a reference to this file.

Yes, that's my next todo.
i think fable should be modified to accept a new argument --fsc-args @path/to/fsc.rsp with the path of a response file (the fsc arguments list saved as multi line text file). If that --fsc-arg is passed, there is no need for guessing args with projectcracker. so it doesnt impact current behaviour, is just an optional arg.

[enricosada]

As a note, an example of final expected for preview5 project atm is like the following.
Take a note that's just a normal .net core sdk F# fsproj, with an additional sdk (Fable), and a fable ref (FablePackage who is read by tools as a normal Reference). So will work ok in ide, because is a normal F# proiect file.
Also NetSTandardLibrary will be implicit (so optional) like TargetFramework probably

<Project Sdk=".NET; F#; Fable">
  <PropertyGroup>
    <TargetFramework>netstandard2.0</TargetFramework>
  </PropertyGroup>

  <ItemGroup>
    <Compile Include="mario.fsx" />
  </ItemGroup>

  <ItemGroup>
    <PackageReference Include="NetStandard.Library" Version="2.*" />
    <PackageReference Include="FSharp.Core" Version="*" />
    <FablePackage Include="fable-core" />
  </ItemGroup>

</Project>

[alfonsogarciacaro]

the dotnet core sdk will support commands to modify the msbuild project programmatically (like forge), for example to add ref is something like dotnet ref add fable-powerpack 1.0.0 ( ref dotnet/cli#4521 ).

The link only talks about project and assembly references so this means we have to add directly the .dll reference. For me that's fine (probably will make things simpler) but it means we won't have a new MSBuild item (FableReference or NpmPackage) anymore.

The example looks very neat! How would you specify the version of the Fable Sdk?

So, to summarize. I will:

• Add a --fscArgs argument to Fable that accepts the path to the response file.
• Add a --fableVersion argument to Fable to warn the user if the installed version is not the expected one.
• Add a fable install command to install npm packages and add a reference to the .fsproj file.

@enricosada, it'd be awesome if you could:

• Add a dotnet fable command that just redirects commands to the locally installed Fable compiler, ideally with the expected version number.
• Create the response file and send it to Fable.

[enricosada]

The example looks very neat! How would you specify the version of the Fable Sdk?

Like Fable/1.2.3. It's not final but likely. Probably will be just Fable but version is in another file at repo level

The link only talks about project and assembly

It's OSS, we can just send a PR to dotnet/cli. Or fallback to create a clitool, anyway not a blocker

@enricosada, it'd be awesome if you could:

Perfect! Fine by me

[alfonsogarciacaro]

@ncave If we move Fable to a pure JS app, I guess it doesn't make so much sense to advance in the integration with dotnet SDK? Will we continue to use Forge or similar to parse .fsproj files?

[enricosada]

@alfonsogarciacaro If you run f# compiler with js instead of .net, this is not required ihmo.

But if fable is integrated with the netcore sdk, every ide who work with the new msbuild fsproj, will work with fable. that's the only good thing.
Because the msbuild will run fable command doesnt matter if that is js or .net. but obv that add netcore as requirement, dunno if is needed.

If you want close i understand, is open by some time.

[ncave]

IMO it will be nice to maintain both dotnet and js versions of Fable, as they're both xplat and serve different scenarios. The Fable JS prototype is not yet ready to parse multi-file projects (unless they're combined into a single file), and it's still very rough and unoptimized.

[alfonsogarciacaro]

Sorry for the late response. @ncave, I think you're right, it's better to keep Fable as a dotnet app for the moment. In the next major version I think we're probably ready to merge the .NET/Mono and netcore versions. Unfortunately, the FCS Nuget package doesn't seem to contain a lib/netstandard1_6 folder yet (even if there's a FAKE target for that), so I'm not sure how building in the CI server and in a machine without dotnet SDK preview2 (FCS still uses project.json) will work. Do you know something about that?

@enricosada I've been playing a bit with the latest bits of dotnet SDK and, although I still like the idea of integrating it with Fable, I'm starting to wonder if that won't be too much work for little benefit. The thing is because of the way Fable works now: CLI options, watch mode, JS API... it would require a lot of refactoring to change the workflow and make Fable be bootstrapped by the dotnet sdk. There're also other things that Fable can do and are not possible with dotnet SDK: like merging several projects to compile them at once or compile .fsx scripts. Also, because the new .fsproj format still uses Compile items, using a simple XML parsing as we're doing now (thanks to Forge) we can provide support both for the new and old formats without little change in Fable code.

About other features of dotnet SDK like adding references or templates, unfortunately, it seems the latest version doesn't allow adding .dll references from the CLI (only Nuget and project references AFAIK) so it's not very useful for us. Also, I couldn't find how the templates work, so I'm not sure if they'll fit our needs (interacting with non-dotnet tools like npm, etc...) and it may be better to create a simple custom template engine. I'm not very convinced of having a FableSDK either, we already have issues with users installing different versions of fable-core and fable-compiler locally or globally, and adding another tool could make things a bit more confusing.

All in all, I think the most interesting part about dotnet SDK for Fable is generating the response file out of an .fsproj, so we don't need to make a guess of the reference locations and also advanced MSBuild features like conditional items can be used (again) with Fable. But this is actually what the FCS project cracker was doing, so I wonder if it should be implemented there instead of being a Fable-specific feature.

There may be also another idea: comparing the FSharp.NET.Core.Sdk.targets with the proof of concept you created for Fable, as far as I can see the only difference is the former calls the F# compiler at the end. Would it make sense to split the CoreCompile target in FSharp.NET.Core.Sdk.targets, so any tool (not only Fable) can call it just to generate the response file without building (e.g. dotnet build /t:GenerateResponseFile)? I guess this would be also very useful for editors.

What do you think?

[enricosada]

So i didnt continued coding last month, but i think about it working on the sdk/msbuild of new fsproj.

the latest version doesn't allow adding .dll references

The old <Reference> works, it's just not used often (because packages are first class)

Would it make sense to split the CoreCompile .. generate the response file without building
like dotnet build /t:GenerateResponseFile

That's what i am trying to do, and that's all the integration fable need most.
But is not easy to share the logic. There are some options (like override fsctask or output property, but neither is perfect). That said project cracker works for all simple proj, but doesnt full evaluate everything (conditionals, etc), and some target are executed only on build. That maybe doesnt matter for fable usage, and project cracker is enough.
Moving forward the same method used by FCS will be used by fable too, that's for sure.
We need eval: fsproj -> compiler args, just not implemented like projectcracker. Maybe will be a lib, a target or something else. If possibile will be embedded inside FSharp.NET.Sdk, so easier to do a dotnet build /t:GenerateResponseFile stable api for consumers (or reference an assembly from sdk of fsproj, so can evolve)

Fable.SDK and advanced usage

Agree with you @alfonsogarciacaro . I was trying some possibilities, but having all fable on js/npm side, make useless use advanced features of sdk.

The only good parts left, afaik, as previous discussions:

• compiler args (project cracker replacement)
• auto include of <Reference /> choosen by fable (so fsproj is slim, and usually changed only for files/defines). That may works also for Compile, Defines too, so ideally the fsproj doesnt need to be changed at all.
• fsproj works with ides, without special knowledge

So a simple fsproj, with all configuration (reference,defines,sources) in fable.config, who doesnt need to be tweaked.

The excluded parts (managed by npm/js side):

• package management
• publish
• fable as .net cli command (dotnet fable)