Flow: Return function of polymorphic high order function is not type checked as expected

[Gozala]

Here is a following example:

/* @flow */

type SourceState = { count: number };
type AppState = { sources: SourceState };

const getSources = (state:AppState) => state.sources;
const multiplyCount = (state:SourceState, x:number) => state.count * x;

export const wrap = <outer, inner, value, settings>
  ( selector: (input:inner, options:settings) => value
  , accessor: (input:outer) => inner
  ) =>
  (input:outer, options:settings):value =>
  selector(accessor(input), options)


export const wrapped1 =
  ( multiplyCount
  , getSources
  );

// Should type check
wrapped1({ sources: { count: 5 }}, 3);

// Should not type check
wrapped1({ sources: { count: 5 }}, 'Not a number');

Flow type checks it fine, but in fact it should reject second call to wrapped1 as string is passed instead of a number.

[Gozala]

/CC @jlongster

[avikchaudhuri]

@Gozala are you sure about the code? wrapped1 is a paren expression, so it looks like you're calling getSources ultimately, which only has one parameter and so only cares about the first arg in the calls. So no problem (yet).

[Gozala]

@avikchaudhuri Oops my bad. Here is correct code:

Here is the corrected code:

/* @flow */

type SourceState = { count: number };
type AppState = { sources: SourceState };

const getSources = (state) => state.sources;
const multiplyCount = (state, x) => state.count * x;

export const wrap = <outer, inner, value, settings>
  ( selector: (input:inner, options:settings) => value
  , accessor: (input:outer) => inner
  ): (input:outer, options:settings) => value =>
  (input, options) =>
  selector(accessor(input), options)


export const wrapped1 = wrap
  ( multiplyCount
  , getSources
  );

// Should type check
wrapped1({ sources: { count: 5 }}, 3);

// Should not type check
wrapped1({ sources: { count: 5 }}, 'Not a number');

Which flow fails to type check, but more importantly complains about missing type annotations (first tow errors) that are in fact there:

export const wrapped1 = ort const wrapped1 = wrap
                            ^ type parameter `outer` of function call. Missing annotation
17: export const wrapped1 = ort const wrapped1 = wrap
                            ^ type parameter `settings` of function call. Missing annotation
7: const multiplyCount = (state, x) => state.count * x;
                                                     ^ string. This type is incompatible with
7: const multiplyCount = (state, x) => state.count * x;
                                       ^ number

And here is the modification that flow type checks with expected error:

/* @flow */

type SourceState = { count: number };
type AppState = { sources: SourceState };

const getSources = (state) => state.sources;
const multiplyCount = (state, x) => state.count * x;

export const wrap = <outer, inner, value, settings>
  ( selector: (input:inner, options:settings) => value
  , accessor: (input:outer) => inner
  ): <model:outer, config:settings> (input:model, options:config) => value =>
  (input, options) =>
  selector(accessor(input), options)


export const wrapped1 = wrap
  ( multiplyCount
  , getSources
  );

// Should type check
wrapped1({ sources: { count: 5 }}, 3);

// Should not type check
wrapped1({ sources: { count: 5 }}, 'Not a number');

7: const multiplyCount = (state, x) => state.count * x;
                                                     ^ string. This type is incompatible with
7: const multiplyCount = (state, x) => state.count * x;
                                       ^ number

My understanding is that for some reason flow want's returned functions from the high order polymorphic function to be also polymorphic or else it complains about missing annotations.

It seems like a bug & it took me and @jlongster several hours to figure out what the actual issues were.

[avikchaudhuri]

I see, OK. Thanks for the interesting examples!

Your analysis is correct. There is a simple explanation but there's no way currently to figure it out from the error messages, sorry. Will fix. But the explanation is as follows.

A polymorphic function is instantiated with some unknown type args when you call such a function. The missing annotations want you to pin down those type args when you're exporting stuff.

By making the returned function explicitly polymorphic in your second example, you're doing the thing Flow wants. But it's important to understand that in your earlier example, you're _not_ returning a polymorphic function. So wrapped1 is not polymorphic: it is a function instantiated at some type outer, inner, value, settings chosen to make that call to wrap typecheck, that's it. With getSources and multiplyCount also unannotated, it's quite possible that the chosen types are fat enough to accommodate both number and string as the second argument to wrapped.

Technically, dealing with nested polymorphism is already hard enough that even languages like ML don't infer it: they only have polymorphism at the top level. In Flow we're dealing with a more expressive system (with subtyping instead of unification), so we don't try to infer polymorphism at any level...it must be made explicit.

Anyway, thanks again for these examples...I'll dig into them more and will probably have more to say later.

[Gozala]

see, OK. Thanks for the interesting examples!

Your analysis is correct. There is a simple explanation but there's no way currently to figure it out from the error messages, sorry. Will fix. But the explanation is as follows.

Yeah better error messages would definitely help a lot. In larger code base pinning down the code causing trouble tends to be very difficult in my experience & can eat up a whole day.

A polymorphic function is instantiated with some unknown type args when you call such a function. The missing annotations want you to pin down those type args when you're exporting stuff.

I find this quite counter intuitive. What I would expect is that if types aren't fully pinned down flow would just treat inner function as a polymorphic one with outer parameters applied as constraints. Which is pretty much what the second example does but by actually writing it out.

What I also find kinda odd is that flow fails even when all of the parameters can be pinned down, here is a modified example:

/* @flow */

type SourceState = { count: number };
type AppState = { sources: SourceState };

const getSources = (state:AppState):SourceState => state.sources;
const multiplyCount = (state:SourceState, x:number):number => state.count * x;

export const wrap = <outer, inner, value, settings>
  ( selector: (input:inner, options:settings) => value
  , accessor: (input:outer) => inner
  ): (input:outer, options:settings) => value =>
  (input, options) =>
  selector(accessor(input), options)


export const wrapped1 = wrap
  ( multiplyCount
  , getSources
  );

// Should type check
wrapped1({ sources: { count: 5 }}, 3);

// Should not type check
wrapped1({ sources: { count: 5 }}, 'Not a number');

So given that selector parameter that is fully annotated multiplyCount and fully annotated accessor parameter that is fully annotated getSources all of the type parameters can be pinned down and there is no real need for wrapped1 to be polymorphic, but that does not seem to be the case. Am I misunderstanding something here ?

I guess second example is also why intuitively I expect flow to figure out what the type of wrapped1 should be as in some instances it could be polymorphic in others it's not. Not sure if I'm explaining it well.

By making the returned function explicitly polymorphic in your second example, you're doing the thing Flow wants. But it's important to understand that in your earlier example, you're not returning a polymorphic function. So wrapped1 is not polymorphic: it is a function instantiated at some type outer, inner, value, settings chosen to make that call to wrap typecheck, that's it. With getSources and multiplyCount also unannotated, it's quite possible that the chosen types are fat enough to accommodate both number and string as the second argument to wrapped.

I guess what I'm failing to understand is why flow can't automatically treat inner functions as polymorphic if type parameters of the outer function aren't fully pinned. In fact I think it would make sense to just always function retuned from the polymorphic functions as polymorphic & just set bounds of it's parameters to outer function parameters. Does that makes sense ?

Technically, dealing with nested polymorphism is already hard enough that even languages like ML don't infer it: they only have polymorphism at the top level. In Flow we're dealing with a more expressive system (with subtyping instead of unification), so we don't try to infer polymorphism at any level...it must be made explicit.

Anyway, thanks again for these examples...I'll dig into them more and will probably have more to say later.

Hmm is that so ? I was under different impression. Have not run into anything like this there maybe due to currying ?

For example here is the same code in Elm

import Html

type alias SourceState =
  { count: Float }

type alias AppState =
  { sources: SourceState }

getSources state =
  state.sources

multiplyCount state x =
  state.count * x

wrap :
  (inner -> settings -> value) ->
  (outer -> inner) ->
  (outer -> settings -> value)
wrap selector accessor =
  \ input options -> selector (accessor input) options

wrapped1 = wrap multiplyCount getSources

state = 
  { sources =
    { count = 3
    }
  }

Doing all the inference more intuitively IMO. You do annotate result of wrap but with a same type parameters and it seems to infer correctly that result is polymorphic. It prints correct value if you add

 main = Html.text (toString (wrapped1 state 3))

And reports the type mismatch when there is a mismatch

main = Html.text (toString (wrapped1 state "Not a Float"))

Detected errors in 1 module.


-- TYPE MISMATCH ---------------------------------------------------------------

The 2nd argument to function `wrapped1` is causing a mismatch.

32|                             wrapped1 state "Not a Float")
                                               ^^^^^^^^^^^^^
Function `wrapped1` is expecting the 2nd argument to be:

    number

But it is:

    String

Hint: I always figure out the type of arguments from left to right. If an
argument is acceptable when I check it, I assume it is "correct" in subsequent
checks. So the problem may actually be in how previous arguments interact with
the 2nd.

[skevy]

@avikchaudhuri out of curiosity, has there been any updates around properly exporting polymorphic functions?

My use case is:

https://flowtype.org/try

As you can see, the flow type in that example works appropriately.

But when I export const actions, and attempt to use it elsewhere in my application, the flow checking doesn't occur as it does when I access actions in the same file.

It seems related to this:

https://flowtype.org/try

I can make ^^ those errors go away by doing:

https://flowtype.org/try

But that's suboptimal of course -- I don't want product developers to have to use the crazy $FnWithArgs type.

[ghost]

Here's a really simple of example of polymorphic curried function which is broken in the current version of flow (0.39.0)

const first = <A> (x: A) => (y: A): A => x;
const second = <A> (x: A) => (y: A): A => y;

// broken
first (5) (1)   // should return 5
second (5) (1)  // should return 1
first ('a') (1) // should be error: 1 is not a string

Errors

5: first (5) (1)   // should return 5
              ^ number. This type is incompatible with the expected param type of
1: const first = <A> (x: A) => (y: A): A => x;
                                   ^ some incompatible instantiation of `A`
6: second (5) (1)  // should return 1
               ^ number. This type is incompatible with the expected param type of
2: const second = <A> (x: A) => (y: A): A => y;
                                    ^ some incompatible instantiation of `A`
7: first ('a') (1) // should be error: 1 is not a string
                ^ number. This type is incompatible with the expected param type of
1: const first = <A> (x: A) => (y: A): A => x;
                                   ^ some incompatible instantiation of `A`

I understand why but I doubt this will be easy to fix. The context of the polymorphic type A is only available for the first function. The returned function (y: A): A => ... does not have concept of its parent's polymorphic type A, so instead it's looking for an object constructed with the (nonexistent) A constructor

example on flowtype.org/try

[szagi3891]

Any plans to fix this error? I'm also annoyed by this bug :)

[nmn]

In @naomik's example:
This looks like an inference bug.

This works:

const first = <A> (x: A): (A => A) => (y: A): A => x;
const second = <A> (x: A): (A => A) => (y: A): A => y;

// fixed
const a: number = first(5)(1);   // should return 5
const b: number = second(5)(1);  // should return 1

const x: string = first('a')(1) // error: x is string | number

@naomik The third case is not a bug as A is just inferred to be string | number.

[nmn]

That said, this works perfectly in Typescript. So we have catching up to do.

const first = <A>(x: A) => (y: A): A => x;
const second = <A>(x: A) => (y: A): A => y;

const x = first(1)(2);
const y = second(1)(2);

// errors as 'a' and 2 don't match types.
const z = second('a')(2);

// this works though:
const z = second(<string | number>'a')(2);

[vkurchatkin]

@nmn I don't think that it's a bug. Flow just can't infer anything containing a type parameter

[szagi3891]

I have this case :

type MenuPropsInType = {||};

type MenuPropsOutType = {|
    menu: Array<MenuItemRecord>,
    menuRoot: Array<MenuRootItemType>,
|};

const mapToProps = (props$: Observable<MenuPropsInType>): Observable<MenuPropsOutType> => {
//...
};

const MenuFn = (props: MenuPropsOutType): React.Element<*> => {
//...
};

export const createRxComponent = <PropsTypeIn: Object, PropsTypeOut: Object>(
    mapProps: (observable: Observable<PropsTypeIn>) => Observable<PropsTypeOut>,
    InnerComponent: (prop: PropsTypeOut) => any,
): ((prop: PropsTypeIn) => any) => {
//...
};

I got error:

222: export default createRxComponent(mapToProps, MenuFn);
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ type parameter `PropsTypeIn` of function call. Missing annotation

But if I use this construction :

const Menu: (props: MenuPropsInType) => React.Element<*> = createRxComponent(mapToProps, MenuFn);
export default Menu;

Then I not have any errors in flow check.

Edit:
When I move create createRxComponent to a separate file then the flow stops reporting any errors. This behavior is also very disturbing because problems are hidden.

[nmn]

@vkurchatkin I used a loose definition of the word 'bug'! The problem is with the inference all the same.

[vkurchatkin]

@nmn what I mean is that it's probably not going to be fixed, because that's just how things work