Flow: Document advanced Utility Types

[gabro]

In the process of learning Flow and adopting it at work, I've often needed "advanced" operators that are completely missing from the documentation.

So far I relied on this blog post (now slightly outdated) http://sitr.us/2015/05/31/advanced-features-in-flow.html and code found in the Flow and React codebases.

I think I'd be nice to have this features explained in the official documentation.

The features I'd like to see documented are:

• [x] [$Keys](https://flow.org/en/docs/types/utilities/#toc-keys)
• [x] [$Diff](https://flow.org/en/docs/types/utilities/#toc-diff)
• [x] ~$Supertype~ (deprecated)
• [x] ~$Subtype~ (deprecated)
• [x] ~$Abstract~ (removed)
• [x] [Class](https://flow.org/en/docs/types/utilities/#toc-class)
• [x] [$PropertyType](https://flow.org/en/docs/types/utilities/#toc-propertytype)
• [x] [* (existential types)](https://flow.org/en/docs/types/utilities/#toc-existential-type)
• [x] [$ObjMap](https://flow.org/en/docs/types/utilities/#toc-objmap) / $TupleMap
• [x] [$ObjMapi](https://flow.org/en/docs/types/utilities/#toc-objmapi)
• [x] [$Shape](https://flow.org/en/docs/types/utilities/#toc-shape)
• [x] [$ReadOnlyArray](https://flow.org/en/docs/types/arrays/#toc-readonlyarray)
• [ ] $Compose
• [ ] $ComposeReverse
• [ ] $CharSet
• [x] [$NonMaybeType](https://flow.org/en/docs/types/utilities/#toc-nonmaybe)
• [x] [$Rest](https://flow.org/en/docs/types/utilities/#toc-rest)

And maybe (they look like internal features though):

• [x] $Exports
• [x] ~$Pred~ (experimental)
• [x] ~$Refine~ (experimental)

Am I missing something?

In order to get the ball rolling, I've sketched a documentation page for "dynamic" enums using $Keys (aka $Enum). I have a PR almost ready that I will submit soon.

Is this something that you guys are already working on?
I'd be glad to help (provided that I understand the features first 😅).

Let me know!

[gabro]

PR submitted, feedbacks are very welcome!

[anmonteiro]

@gabro I think $Exact is also missing docs, as well as its literal ({| |}) notation?

[gabro]

@anmonteiro yes, it's on my radar but I didn't include it there as I wasn't sure $Exact is an internal-only representation of {| |} or not. Along the same lines, we could also document $All $Either and similar utility types that have an ad-hoc syntactic sugar.

Long story short, are $Exact, $All, $Either (and others?) also intended for client's use, or are they just internal representations? This is a question for the Flow team, though. /cc @mroch @thejameskyle

[gabro]

@anmonteiro about the documentation for {| |}, yes, it's missing, although I'm not sure it would fit this issue, which is about "advanced" features, named "Utility Types" in the first related PR. Maybe we should open another issue?

[anmonteiro]

@gabro IMO I agree it's a different concern so another issue is necessary. I should have clarified my initial intent, which was to bring up the fact that it wasn't documented, not to include in a PR for this one.

[STRML]

Can anyone elaborate on usage of $Abstract? We were subclassing React.Component and had a surprising number of bugs disappear (and new, good type errors) once I realized the builtin uses $Abstract<State> and $Abstract<DefaultProps>.

[gabro]

@STRML from what I was able to "reverse engineer", it looks like $Abstract imposes a constraint on a subclass, forcing it to provide that field. Here's a few examples that helped me clarifying the concept:

class MyAbstract<T> {
  foo: $Abstract<T>
}

class MyConcrete extends MyAbstract<string> {} // error, foo is missing
class MyConcrete2 extends MyAbstract<string> {
  foo = 'bar' // ok
}
class MyConcrete3 extends MyAbstract<string> {
  foo = 42 // error, foo must be a string
}

class MyAbstract2<T> {
  foo: T
}
class MyConcrete4 extends MyAbstract2<string> {} // no problemo

From the examples above, the most notable difference between T and $Abstract<T> is that the former allows for a missing field, whereas the latter requires the subclass to provide it.

I would love if anyone from the Flow team could confirm my "hypothesis" (so that we can document this!)

[STRML]

Interesting insight. It's odd, though, that you're not required to actually define static defaultProps or state when extending React.Component.

[gabro]

@STRML I think you are required to, if you specify a type for it. Meaning

class MyComponent extends React.Component { } // ok
class MyComponent extends React.Component<{ x: string }, any, any> { } // error
class MyComponent extends React.Component<{ x: string  }, any, any> {
  static defaultProps = { x: 'foo' };
} // ok

Here's some example from the tests: https://github.com/facebook/flow/blob/master/tests/new_react/bad_default_props.js

[STRML]

Gotcha. So if you just don't specify the type parameters at all, there isn't a difference? Strange.

Still, I wonder why updating our React.Component extension to use $Abstract<DefaultProps> and $Abstract<State> (from non-$Abstract versions) uncovered so many previously-ignored type errors. There must be more to it.

[gabro]

So if you just don't specify the type parameters at all, there isn't a difference?

I think as long as the type is unbound anything is accepted, including void, i.e. missing.

But again, I'm just guessing here. Either I'll learn some OCaml and start reading the source code, or someone from the Flow team can shed some light! 😄

[gabro]

Just to clarify my void point before:

class MyAbstract<T> {
  foo: $Abstract<T>
}

class MyConcrete extends MyAbstract<void> {} // ok!

[gabro]

@anmonteiro I just opened a new issue for tracking the {| |} documentation. Refer to https://github.com/facebook/flow/issues/2524

[gabro]

Adding $ObjMap to the list. I really can't wait for #2465 to be merged, so that we have a scaffold to work on!

[STRML]

Re: $Pred/$Refine, found this:

  $Pred<N> is an "abstract predicate type", i.e. denotes a (function) type that
  refines N variables. So if `cb` is a function, then it should be refining
  exactly N argument. It is abstract in that we do not need to specify:
  (a) which variables are going to be refined (just the number), or (b) what
  exactly the refinement (predicate) is going to be.

  $Refine<T,P,k> is a refinement type, that refines type T with the k-th
  argument that gets refined by an abstract preficate type P.

[STRML]

@andreypopp Could you provide any information on $ObjMap, since it appears you've used it in validated? I've had no luck figuring it out so far.

Edit: Got something going, it appears this doesn't throw an error if you rebind the var (so mirrored.bar = 'foo' is legal, but casting it to a different literal is not). It's definitely fiddly to use.

[gcanti]

@STRML some infos here https://github.com/facebook/flow/issues/2674

[cqfd]

Another +1 for documentation for $ObjMap and $TupleMap. They're super cool and I wish I had found out about them sooner!

I think a nice motivating example would be Flow's type for Promise.all:

```.jsx
static all>(promises: T): Promise<$TupleMap>;
declare function $await(p: Promise | T): T;

You could then use `$ObjMap` to type something like Bluebird's `Promise.props` (basically all but for an object of promises):

```.jsx
static props<Elem, T: { [prop: string]: Elem }>(promises: T): Promise<$ObjMap<T, typeof $await>>;

[STRML]

Hoping the core team will place more importance on this. I understand some of these features are subject to change, but the community could help shape their development.

[STRML]

That's appreciated, but largely incomplete, especially compared to the post from 2015.

[STRML]

That particular one is actually in the OP. But yes, there is a lot of missing information. I have a PR open but it has not gotten any attention.

[STRML]

One more to add to the OP: $Shape, which is mentioned briefly but not explained.

$PropertyType<T, x>, *, Class<T>, and $Exact<T> can be marked as handled.

Can anyone explain the difference between $ObjMap and $ObjMapi?

[cr0cK]

@STRML Check the comment of that commit: https://github.com/facebook/flow/commit/5a05018b01e0eb1537598df6ffe1c57233140be5
Btw I'm looking too for documentation about all that "magic types". For now, I'm trying to understand by looking code of @gcanti and it often misses hints or comments to understand how it works or how it should be used.

I have some internal libs written with my co-workers and we'd like to write declarations but we really miss this part of the documentation.

[gabro]

I finally found some time to work on this. Here's a PR to document ObjMap https://github.com/facebook/flow/pull/3589

I plan to add ObjMapi and TupleMap soon, since they're very similar and can be explained by comparing them to ObjMap

[STRML]

To add to this list: $ElementType<T, string> as mentioned here. An example here (set branch to master). Added in 968210c5887b5bdd47d17167300033d1e1077d1a.

[Chaoste]

$Abtract is still not able to handle abstract classes, is it? Example usage

I get the following errors:

 isValid: $Abstract<() => boolean>
                        ^ function type. This type is incompatible with
7:   isValid: $Abstract<() => boolean>
                        ^ undefined. Did you forget to declare function type?

[nmn]

@Chaoste There is a small difference in how functions are typed, which is what is causing some of the errors. See example here.

[zaaack]

And $ReadOnlyArray

[niieani]

Two more recent ones are $Compose and $ComposeReverse from this commit.

[lll000111]

Maybe I have the wrong idea how things work, but my crude notion of how this is supposed to work is that the requirement to have documentation should be part of the commit-review process?

No commit/merge of undocumented features. Instead of trying to hunt down someone who might be able to shed some light on those features afterwards. Including a checkbox "for internal use only", "not sure if public API", "intended for public API", etc.

The effort of documenting something is lowest for the person who implements it - at the time they do so (locality in space as well as in time :-) ).

[jcready]

Indeed. This isn't just a fun side project anymore, people are using this in production. And if you're trying to compete with TypeScript, it won't matter how sound your type system is if you aren't providing up-to-date documentation.

[STRML]

I agree @lll000111 - while the Flow team is doing really great work, it feels to me like there are two disconnected sides to development - development within Facebook, which appears to have its own discussion and issue tracker, and what is released here to us in public.

It is proving difficult to get the original authors of these features to comment on how they might be used, if they are meant to be public or not, and if they are going to be improved or are intended to work as they work today.

A few hours from someone on the core Flow team toward improving the $ types would go a long way.

[niieani]

I think the right person to address the above mentioned concerns is @calebmer.

[CMCDragonkai]

The $Abstract appears to be removed so it is no longer supported.

[k]

Can we add $Compose and $ComposeReverse to this list as mentioned by @niieani above?

[bradennapier]

Looks like $Rest should be added:

/* @flow */

type ABC = {| a: 'A', b: 'B', c: 'C' |};

type WithoutAB = $Rest<ABC, {| a: 'A', b: 'B' |}>;

type ObjWithCOnly = {| c: 'C' |};

function test<O: ABC>(o: O) {
    const {a, b, ...rest} = o;

    (rest: WithoutAB);

    (rest: ObjWithCOnly);
}

test({ a: 'A', b: 'B', c: 'C' })

[bradennapier]

Also, looks like master will be adding $ReadOnly which will be great to have. I am guessing it simply converts {| a: string, b: number |} -> {| +a: string, +b: number |}

[bradennapier]

There is also $CharSet which is fairly straight forward.

type ABorC = $CharSet<'ABC'>;

// $Works
('ABC': ABorC);

// $Works
('CBA': ABorC);

/*
  // $ExpectError
  string literal `CCBA`
  This type is incompatible with
  character set `ABC`
    ---
    `C` is duplicated
    ---
*/
('CCBA': ABorC);

[villesau]

To be honest, i still can't understand difference of $ElementType and $PropertyType, see:

https://flow.org/try/#0C4TwDgpgBA8gRgKygXigbwFBSmATgezAC50A7AVwFsSLK4JcBfDZjACjVpIEYAmAZkYkAJAFEANhEoRSwACrgIAHngIANFADkeQpoB8ASgDc7TlR4ChUYQAUCkXKAWQViDdvv7jQA

Both has same functionality?

[jcready]

@villesau $PropertyType can only take an object type as the first parameter and only a string literal as the second parameter. $ElementType on the other hand can take object or array types as the first parameter and almost any flow type as the second parameter. I'm not sure what the point of using $PropertyType is to be honest since $ElementType seems to do everything it can, but more.

type Tuple = [null, true, 'bar']
type Obj = {|
  foo: 1,
  bar: false
|}

type propfoo = 'foo' | 'bar';

declare var x: $ElementType<Obj, propfoo>;
declare var y: $ElementType<Tuple, 2>;
declare var z: $ElementType<Tuple, any>;

(x: 1 | false);
(y: 'bar');
(z: null | true | 'bar');

[LoganBarnett]

@gabro per @zaaack's comment - can we add $ReadOnlyArray to your list, or is that beyond the scope of your PR (or PRs)?

Also, it looks like $SubType, $SuperType, $TupleMap, $ObjMap (although not $ObjMapi) are in the published docs so we could check those off.

Edit: Whoops! Somehow this got posted twice.

[gabro]

@LoganBarnett thanks for the ping. I've updated the post to include $ReadOnlyArray and $ObjMapi and I've checked off the ones that are already in the doc.

Unfortunately I won't be able to contribute to the docs myself in the short period, but I'm glad to see things are moving :)

[villesau]

$SubType and $SuperType doesn't have proper documentation yet though.

[julienw]

The doc about utility types has just been updated => https://flow.org/en/docs/types/utilities/
We still miss some like Shape.

[LoganBarnett]

@villesau blugh that was pretty careless of me to miss that. Thanks for catching it!

@gabro it's much appreciated that you're keeping the ticket up to date though. I imagine any kind of bookkeeping we do immensely helps people when they can function as active maintainers. To the @villesau's point above, $SubType and $SuperType are documented with the exact text "Work in Progress" which is kind of like the honorable mention equivalent of documentation. Can we uncheck those two boxes please?

[gabro]

@LoganBarnett sure, done!

[zen0wu]

It seems that $Abstract utility type is removed, so we're back to square one on this topic!

[bitpit]

Man this is disheartening coming from typescript

[lll000111]

@bitpit It's worse when you compare the issues lists of both TypeScript and of Flow. One of those two projects gets issues closed, the other one just collects them.... (not that the list of open issues is any shorter though) and we Flow users are mostly talking among ourselves here in the issues comments. I filed only one issue for TypeScript and had Anders Hejlsberg himself reply to it, and it was not an especially noteworthy issue at all.

[zen0wu]

@lll000111 I had a lot of TODO (flow/#xxxx) in my codebase, but this one is my favorite.
Context: This is the type for the stack in componentDidCatch feature in React 16.

// TODO (flow/#5139): It's pretty lame that TypeScript has the stubbed type but not flow itself
type ReactErrorInfo = {
  componentStack: string,
};

[forticulous]

Since I didn't see it anywhere, we should add $NonMaybeType to the list of undocumented advanced types

[mrkev]

Done 👍

[sibelius]

should we document $Pred and $Refine as well?

[jamesisaac]

Nearly all of these are documented (or deprecated) now, mostly under Utility Types, but I've added links to the original list anyway for reference.

$ObjMapi has a PR waiting, so it looks like all that's left is $Compose, $ComposeReverse and $CharSet.

$Pred, $Refine and $Exports do sound like they're internal types not meant for public use -- maybe someone on the Flow team could confirm? cc @nmote @jbrown215

[nmote]

$Pred, $Refine and $Exports do sound like they're internal types

Honestly, I'm not even sure what these do, so my guess is yes. Maybe @panagosg7 could weigh in (looks like he authored the file that you linked)

[bradennapier]

I learned about $Pred and $Refine from https://medium.com/netscape/secret-flow-types-86b2ebb30951 then by reading the documentation in the actual code which I believe was fairly detailed. However, not sure I ever used it anywhere in the end - this was ages ago now heh :-P

Update: Not finding those comments in the codebase now :-D maybe i hallucinated them hah

[panagosg7]

Indeed, $Pred and $Refine are purely experimental. They're mostly proof of concept for my internship project at the time. The API they expose has several holes that I haven't gotten to seal, since priorities have shifted since.