Fable: [Help Wanted/Design] Improve TypeScript integration

[matthid]

What?

Basically, I want to improve the interop with TypeScript and asking for help and directions.

This means extending the fable-compiler in two areas:

1. Emit TypeScript bindings (we currently write them by hand)
2. Consume/Import TypeScript (we currently use ts2fable for this)

Please feel free to correct me throughout the post, suggest alternative implementations or even point out why this is a bad idea. Any help regarding the design, implementation or else is welcome. Feel free to connect privately via Twitter or Slack (or Mail).

Why?

The linked related issues (see the end of this post) should give a couple of reasonable use cases. But in particular:

• After "1." it is easier to publish fable-based libraries on npm without any additional manual work.
• "1." should allow import "./MyFile.fs" in TypeScript/Javascript with proper IDE support. (It is not exactly clear what additional thing we would need to do here for the IDE to pick up the typings)
• "2." Would make it a lot easier for newcomers to use packages in the npm ecosystem
• "2." Might allow us to reduce our work of maintaining and keeping the typings updated.
• (Future discussions, just thinking our loud) After 1 + 2 we could potentially publish packages on npm. For F# tooling we might still need to include a reference assembly (or just the .net assembly, considering WebAssembly...). There are however a lot of other issues to be solved. Just putting this here to put it up for discussion if this is what we want eventually.

How?

I'd like to talk about 1 only for now (and extend the post later)

Emit TypeScript bindings

Looking at the available APIs and the related discussions I feel like the best bet is to use the TypeScript API as there are online tools which make is easy to understand and work with.

I have played a bit with the code base and noticed the following:

• We could just use the TypeScript-API in fable-compiler-js, we just need a couple of bindings or use unsafe calls.
• However, in regular .NET based fable we cannot use the TypeScript API, so we need a intermediate serializable datastructure for type declarations built from Fable.AST. It will look similar to Fable.AST but only type-specific stuff (all Expression stuff will be removed)
• I haven't figured out what the best way is to inject TypeScript definitions into webpack via fable-loader, but as a first step I would just output the .d.ts files somewhere (alongside the .fs file for example).

Yes it probably is a bit of work but all it all it sounds doable to me. In practice:

• We could start with a quite minimal implementation only supporting simple interfaces (for example) and using any for everything else
• We probably want to make this opt-in until we are more confident
• We can extend this feature for feature (ie typings will become better over time)

Consume/Import TypeScript

I will expand this section later or throughout this discussion. But my current ideas are:

• Create a "build-in" type-provider type MyTypings = TscImport("typings.d.ts")
• Write .fs files, for example based on attributes -> add to project file via globbing

Related issues

This issue is a continuation of:

• https://github.com/fable-compiler/Fable/issues/104
• https://github.com/fable-compiler/Fable/issues/232
• https://github.com/fable-compiler/Fable/issues/805
• https://github.com/fable-compiler/Fable/issues/1159

[Luiz-Monad]

The other day I was working on making ts2fable work better translating types. But I stopped after I started hitting problems with the need for type tagged unions. https://github.com/fsharp/fslang-suggestions/issues/538
Also I didn't find a good way to represent the Pick row polymorphism with F# column polymorphism. They don't quite fit together for a 1:1 translation of types.

[Luiz-Monad]

Perhaps we could do the opposite, convert all the TypeScript AST to F# AST. [insert evil mad computing scientist laugh]

[alfonsogarciacaro]

Thanks a lot for this detailed issue @matthid. As we've talked some types, it's true I'm a bit skeptical about the Typescript integration because as @Luiz-Monad says, they type systems of both languages have differed quite a bit. But I'd love to be proven wrong and see how we can take advantage of .d.ts declarations.

I can see you've done your homework and listed the older issues around the topic so I don't need to repeat the info... or actually look for it myself because I've already forgotten it :) But I'll still try to add some comments as I remember things. For now two quick notes:

• There's already work in progress by @ncave to bring back the annotations in the Babel AST #1615. Babel can now parse Typescript, but I don't remember if it works the other way around (annotations in Babel AST used to emit Typescript).
• About importing F# files into Typescript, I've noticed the problem is Typescript doesn't let you declare a non-js module with a relative path. There's a note about this now in the docs.

[matthid]

Thanks!

But I stopped after I started hitting problems with the need for type tagged unions.
Also I didn't find a good way to represent the Pick row polymorphism with F# column polymorphism

As we've talked some types, it's true I'm a bit skeptical about the Typescript integration because as @Luiz-Monad says, they type systems of both languages have differed quite a bit.

Yes, I think this is indeed a problem, but my current thinking is this is only really a problem for "2." not for "1." (or do you know any examples where this is a problem for "1."?). In general it feels like the TypeScript type system is more powerfull than the F# one. This most likely a general problem in the ecosystem as TypeScript just tries to "type" existing javascript. So this basically means we have problems mapping existing code to our type-system.

To solve this for "2." we are not lost either:

• Long term: Openeing suggestions like you did, but we also need to lobby the discussions to stay useful for fable (as you can see in https://github.com/fsharp/fslang-suggestions/issues/538)
• Short term: We can map unknown types to "obj" or a common base-type of the union.
• Mid term: We are a compiler so we could do code generation and emit accessors similar to how we would write them manually by hand. But we need to think about compatibility for this.

There's already work in progress by @ncave to bring back the annotations in the Babel AST #1615. Babel can now parse Typescript, but I don't remember if it works the other way around (annotations in Babel AST used to emit Typescript).

I did some research with google but I cannot figure out what the benefit of this is. If babel cannot write .d.ts files out of this what is it good for? Maybe you or @ncave can explain?

About importing F# files into Typescript, I've noticed the problem is Typescript doesn't let you declare a non-js module with a relative path. There's a note about this now in the docs.

Yes I think we need a bit of fiddling around to make that import work flawlessly but in worst case scenario we can also generate a .ts file which imports the declaration and proper .fs import and then we can tell people to import .ts instead of .fs?

[MangelMaxime]

but I don't remember if it works the other way around (annotations in Babel AST used to emit Typescript)

If I remember correctly, Babel only supports compiling TypeScript into JavaScript. It doesn't generate TypeScript.

At first Fable Conf, at least it was explained that they wanted to support TypeScropt -> JavaScript transpilation and do a "better job" than TypeScript compiler because of all the Babel ecosystem, configuration and optimisation.

[alfonsogarciacaro]

In the first attempts to generate .d.ts files I used babel-dts-generator that was capable of extracting the annotations from Babel AST to create the declarations. But it seems the plugin is not supported now :/ Another option would be to output the annotations as JSDoc comments, as Typescript can also use them to provide intellisense.

If I'm not mistaken, the purpose of @ncave was to make Babel output the type annotations as comments, then do another pass to remove them and finally see if the result was compatible with one these Typescripts subsets targeting WebAssembly like AssemblyScript. Yes, black magic as usual :wink: But if in both cases the purpose is to include type annotations in the output it will be a chance to take two birds with one stone.

[matthid]

Another option would be to output the annotations as JSDoc comments,

I have seen that suggestion, but I doubt it would we easier than writing TypeScript definitions via API. On the other hand, if we get them 'for free' from babel that would be another story and probably the easier approach.

Question is how this fits libraries? I assume we would suggest to include comments in the bundle?

[ncave]

@alfonsogarciacaro

• We can perhaps merge #1615 as is, it's behind a compiler option so it's non-intrusive. It can be nice to let people play with it on the REPL, if the option is exposed in the UI.
• #1615 hasn't changed much in the last year (just rebased), but it already gets you pretty far in outputting Type Annotations (and more can be added easily).
• The first step after that would be to make sure the fable-library compiles to something the TypeScript compiler can accept.
• One of the first type annotations that needs fixing is the function parameters, need to be uncurried to match the uncurrying at the call site.

Side Note:

• I was planning to use that PR to generate types for compiling with AssemblyScript. Unfortunately that project hasn't progressed as much as I hoped in the last year. It got reference counting GC, but still lacks support for some basic features like closures and iterators (perhaps because webassembly itself is slow in adding the required features to support that out of the box).

I hope this changes in the future, but in the mean time targetting TypeScript should be much easier.

[matthid]

The first step after that would be to make sure the fable-library compiles to something the TypeScript compiler can accept.

I hope this changes in the future, but in the meantime targetting TypeScript should be much easier.

Not sure what these mean. Does this mean the way forward is to forward the JSDoc-formatted output into the TypeScript compiler, which will itself write the .d.ts files? Or do you mean writing .d.ts files from fable itself? Can you please clarify @ncave ?

Or put differently: I'm open to invest some time into this problem, however after all these discussions I'm still not sure what our best bet is at this time?

[ncave]

@mattid Adding type annotations in the Babel AST generates types in the Babel output directly. It's just a matter of completeness (output the correct type annotations for all corner cases). For example:

let rec factorial n =
    if n = 0 then 1
    else n * factorial (n-1)

let iterate action (array: 'T[]) =
    for i = 0 to array.Length - 1 do
        action array.[i]

let rec sum xs =
    match xs with
    | []    -> 0
    | y::ys -> y + sum ys

compiles to (with type annotations turned on):

export function factorial(n: number): number {
  if (n === 0) {
    return 1;
  } else {
    return n * factorial(n - 1) | 0;
  }
}
export function iterate<T>(action: (arg0: T) => void, array: Array<T>): void {
  for (let i = 0; i <= array.length - 1; i++) {
    action(array[i]);
  }
}
export function sum(xs: any): number {
  if (xs.tail != null) {
    const ys = xs.tail;
    const y = xs.head | 0;
    return y + sum(ys) | 0;
  } else {
    return 0;
  }
}

As you can see, it's not perfect and there is some work to do (quite a few types are just stubbed as any for now), but IMO you can get quite far with that.

[matthid]

@ncave And I guess TypeScript can consume annotated JavaScript ootb? Or do we need to process this further?

[ncave]

@matthid The above is valid TypeScript produced by Babel, just rename .js to .ts.
But as I said, the PR needs some love to add the proper types, it's hasn't really changed much since @alfonsogarciacaro first added type annotations long, long time ago in Fable 1.

[matthid]

Will take a closer look at the weekend, thanks for clarifying

[alfonsogarciacaro]

I'll soon create a next branch for the next major release to merge #1839. We could also use it to merge #1615 and experiment with it :)

[maxdeviant]

Just wanted to chime in and say that I've been very interested in introducing Fable at my work. We have a 200k+ LOC TypeScript codebase, so having some level of TypeScript integration in Fable would be _huge_.

I'm primarily interested in having Fable emit TypeScript bindings so that it's easier to incorporate a Fable library into our existing TypeScript codebase.

[chrisvanderpennen]

I've been tinkering with some of the output of ncave's recent work, trying to find a way to build TypeScrypt types so that F# unions are accurately typed and can be discriminated with a switch statement without breaking the existing object structure. Is this the right place to post some ideas?

[ncave]

@chrisvanderpennen Sure, why not, if it's related. Or you to open a separate discussion issue, if you want.

[chrisvanderpennen]

Moved to #2096

[ncave]

@chrisvanderpennen Thank you for the detailed explanation, it's probably best to convert this into its own issue [feature request], as it probably goes a bit beyond the initial scope of just adding types.

[chrisvanderpennen]

Sure, I'll create one and edit the above to point to it so it isn't cluttering this discussion.

[AngelMunoz]

So far I think the idea is to avoid javascript/typescript but what if you can embrace it?

I recently worked with [Bolero] which is just webassembly, and the way it does javascript interop is by doing function invocations

https://github.com/AngelMunoz/Mandadin/blob/master/src/Mandadin.Client/Views/Notes.fs#L79

// next version of blazor will actually change
// to import the whole module (store the ref) then invoke the function
// instead of the actual Global Namespacing
Cmd.OfJS.either js "Namespace.MyFn" [||] Success Error

and in the javascript side, I still have to write code by hand

https://github.com/AngelMunoz/Mandadin/tree/master/src/Mandadin.Client/wwwroot/js

and include my js libraries, if this grows big enough I believe you would need to bundle those files and dependencies anyways

Fable already uses Webpack and it's just javascript typescript/javascript files, in the end, at least that's what I think.
The typings/dependencies are just an "npm install" away.

// interop/my-file.ts
// to be included in the final fable bundle
import { libraryFn } from 'the-library-i-wont-write-bindings-to'
// hide the library interop/specifical JS work
function parseThingsAndWorkWithLibraries() { /* ... */ }
function imDoingStuff(someParam)  {
    // code 
    let someParam = someParam['something'] = myfn();
    const parsed = parseThingsAndWorkWithLibraries(someParam)
    return return  { parsed }
}

// export only the F# interop bits
export async function myInteropFn(paramA: string, paramB: number): { my: string, fsharpType: boolean }} {
    try {
        const [unshapedResult, anotherResult] = await Promise.all([
            libraryFn(paramA, paramB), imDoingStuff(paramA)
        ]);
        return { my: unshapedResult.my_value, fsharpType: anotherResult.secondValue };
    } catch(err) {
        return Promise.reject(err.message);
    }
}

and could be consumed in a similar way like this

// in the fable code somewhere 
[<ImportMember("my-file")>]
let myInteropFn(params: string * number ): {| my: string; fsharpType: boolean |} = jsNative

Cmd.OfJS.either js myInteropFn ("value", 10) Success Error

My thought process here was that if Fable could implement an option for interop in a similar matter, the complexity of js interop would be at the F#/JS boundary, not in the lack of hands to invest in tooling/bindings

now, this would be the "Worst Case" meaning that you would only resort to this if the library is really big enough for yourself or there's no effort on the community to write some bindings the safety will always be in the F# side

I believe most of the time you don't really need external libraries and the popular ones might be covered already

anyway... this is just an idea I had when working with Bolero I believe Fable has a better chance to improve that interop model than Blazor/Bolero since the packages already live in npm, there's too few browser ready builds of libraries for Blazor/Bolero to work, in the end, I feel they will still resort to bundling in one way or another

[alfonsogarciacaro]

Thanks a lot for your comments @AngelMunoz! I'm not sure I understand, there are already several ways to interop with JS code from Fable either in a typed or untyped manner, what are you specifically looking for? https://fable.io/docs/communicate/js-from-fable.html

[AngelMunoz]

my comment is about third party library integration, that is complicated to automate (ts2fable) and may be prone to errors, also the only other alternative is creating bindings for third party libraries and sometimes there's just not enough hands hence the need to improve typescript integration, or that's what I understood from the issue thread.

the summary would be the following
if Fable includes any user authored javascript/typescript bundle within the same app you should use the js from fable usual mechanisms without having to write bindings for each library, the bindings would be for your specific code

maybe this is a different issue and I'm not catching the idea

[AngelMunoz]

@alfonsogarciacaro
following from yesterday this is what I meant
https://github.com/AngelMunoz/fable-plus-typescript-files-poc/blob/master/src/App.fs#L8
https://github.com/AngelMunoz/fable-plus-typescript-files-poc/blob/master/src/tsfiles/interop.ts#L18

this doesn't addresses point 1

Emit TypeScript bindings (we currently write them by hand)

but it provides less friction for point 2 of the issue

Consume/Import TypeScript (we currently use ts2fable for this)

it of course brings it's own set of issues, the first one I can think of is safety, the typescript type system can be really good but needs user reinforcements unlike F# which is safer by default

[alfonsogarciacaro]

About point 1, @ncave is working on that, although work is a bit on hold until we release a stable Fable 3. Point 2 is about consuming directly typescript files in a type-safe manner with bindings (or with auto-generated bindings on the fly). I think this is complicated but could be done with a type provider or a tool for code generation.

In any case, as commented above, it's already possible to consume Typescript or JS files by using dynamic operators or writing some ad-hoc bindings (I do that all the time). No need for any additional mechanism.

[CedricDumont]

@alfonsogarciacaro : when you say point 1 is in progress, does this mean usage of "--typescript" in fable 3 ? Is that what you are talking about ?
if yes, it seems this flag does create real typescript code (not only declarations)
my goal :create a library in Fable which I would like to use in another library (that second one in typescript). when using the "--typescript" flag I get some errors

import { decimal } from "./.fable/fable-library.3.0.1/Decimal.js"; // error : ... has no exported member decimal ...

I understand it's a work in progress but could you add a link to the work?
Where should we add issues regarding that specific feature?
what is the correct way currently to use a Fable library in another typescript library ?

[alfonsogarciacaro]

@CedricDumont yes, that's right. Although I must confess that I haven't worked at on this feature 😅 so I don't really know what's the current status. @ncave probably can answer to that better. IIRC the fable-library imports do give issues because atm we're not packing the fable-library files compiled to typescript. We can try to solve that but I'm not sure if there're other issues pending. If we want to give an impulse to this feature, we should try to make the tests run when compiling to Typescript.

what is the correct way currently to use a Fable library in another typescript library ?

Right now, basically writing the .d.ts declaration yourself for the Fable methods you're consuming from TS. I've found the following configuration to work:

App.fs
App.fs.js # generated
App.d.ts # manually written

For example, if your App.d.ts declaration contains the following export (corresponding to an actual export in App.fs.js):

export function foo(x: number): {
    data: string[]
};

You can consume it from Typescript like this:

import { foo } from "./App"

const result = foo(5);

Note the extension is omitted in the import. You need to edit your webpack.config.js so Webpack looks for files with .fs.js extension in this case:

module.exports = {
    resolve: {
        extensions: ['.fs.js', '.mjs', '.js'], // Add other extensions you may want to use
    },
    ...
}

Note this is also useful to consume Fable code if you're using just Javascript with the // @ts-check statement.

[ncave]

Here is the current state of the TypeScript support, based on this comment:

As far as progress goes, we were able to compile fable-library to strict TypeScript in Fable 2.
In Fable 3, we had a bit of regression because of the many changes, but it's getting close again.
After that, the next big goal would be to compile all tests to strict TypeScript, but that has a much bigger scope.

So we have a small hump to go over first, and a bigger one after that, but hopefully it can be semi-usable after the first, if we bundle the TS version of fable-library with the Fable compiler. Opinions and contributions are welcome, build steps for TS fable-library are outlined in the comment mentioned above.

[alfonsogarciacaro]

Closing as there's no work happening at the moment in this direction, please reopen if someone wants to contribute to the TS integration.