Io-ts: We've lost some use cases with the new variance constraints

[sledorze]

Sorry for being that late to react on it..

Whereas the addition of Serialize brings a lot on the table, there's some use cases that can no more be represented with the new incarnation of io-ts.

One can no more use an io-ts type purely for validation and assign it to a wider io-ts type.

Was compiling fine before

export let x: t.Type<'a' | 'b'> = <any>1
export let y: t.Type<'a' | 'b' | 'c'> = <any>1

y = x // Ok
x = y // Error

No more possible (for good Variance reasons)

export let x: t.Type<any, 'a' | 'b'> = <any>1
export let y: t.Type<any, 'a' | 'b' | 'c'> = <any>1

y = x // Error
x = y // Error

A solution to this would be to be able to separate the Validation and Serialization and join them on need.

I'm guessing that's also the motivation behind Reads/Writes and Format of Play API:

https://www.playframework.com/documentation/2.6.x/ScalaJsonCombinators

[sledorze]

Proposal:

Define those:

export declare class ValidateType<S, A> {
    readonly name: string;
    readonly is: Is<A>;
    readonly validate: Validate<S, A>;
    readonly '_A': A;
    pipe<B>(ab: ValidateType<A, B>, name?: string): ValidateType<S, B>
}

export declare class SerializeType<S, A> {
    readonly name: string;
    readonly serialize: Serialize<S, A>;
    readonly '_S': S;
    pipe<B>(ab: SerializeType<A, B>, name?: string): SerializeType<S, B>;
}

export const validator = <I, O>(x: t.Type<I, O>) : ValidateType<I, O> => x
export const serializer = <I, O>(x: t.Type<I, O>) : SerializeType<I, O> => x

Then rework validate to use a ValidateType only and serialize to use a SerializeType.

This would help making those use cases possible again:

// Values choosen to test variance:

let x: t.Type<'a' | 'b', 'a' | 'b'> = <any>1
let y: t.Type<'a',  'a' | 'b' | 'c'> = <any>1


let r = validator(y)
r = validator(x) // Ok

let s = validator(x)
s = validator(y) // KO


let q = serializer(x)
q = serializer(y) // Ok

let t = serializer(y)
t = serializer(x) // KO

[sledorze]

For the impatient (like me) which needs a solution right now.. :

export const validateOnly = <I, O>(
  value: I,
  v: ValidateType<I, O>
): either.Either<t.ValidationError[], O> => v.validate(value, t.getDefaultContext(<t.Type<I, O>>v))

[sledorze]

@gcanti, Iterating on my problems: I was matching on a structure of io-ts Type from their '_tag' property (ex: generate testCheck Generators).

Gist worth thousand words:
https://gist.github.com/sledorze/a4ffa850c1ab71314b4fd0d3d480304e

as I'm matching on those, I needed to represent the (recursive) type of acceptable Types to the function, this led me to open this issue (for that reason and other use cases) because one can no more pass that function a type that is not compatible from a serializable standpoint ..

Now that I've found out a workaround (?) to that variance issue, I'm realising I'm loosing the '_tag' information which I use to match the structure.

It starts to become a very involving workaround and I'm wondering if I'll not diverge a lot from the library, so asking here for either if the variance issue could/would/will be addressed and 'how'

[gcanti]

generate testCheck Generators

@sledorze That's interesting. It's hard to understand your issues without viewing the actual code though, is there a public repo?

[sledorze]

@gcanti the lib is not public but using the gist and pasting this will show the problem:

    const schema = t.interface({
      a: t.string,
      b: t.union([
        t.partial({
          c: t.string,
          d: t.literal('eee')
        }),
        t.boolean
      ]),
      e: t.intersection([
        t.interface({
          f: t.array(t.string)
        }),
        t.interface({
          g: t.union([t.literal('toto'), t.literal('tata')])
        })
      ])
    })

    const schemaGen = toGen(schema) // issue..

btw I've validated we could release a public version of it (if it survives io-ts 0.9.x of course)

(testcheck is this lib: https://www.npmjs.com/package/testcheck)

[sledorze]

@gcanti as a matter of focus, the subject of this issue is not the use case I'm referring to, I can find another, indirect, way to extract the structure.
I'm however thinking it may help to be able to separate the validation / serialisation for variance purpose.

[gcanti]

The issue here is that currently some runtime types classes are too smart. For example InterfaceType

export class InterfaceType<P extends Props> extends Type<any, InterfaceOf<P>> {
  ...
}

Here Props is used to compute the A type, triggering variance errors.

In order to allow your use case (and many others I'm interested in) it would be better to make InterfaceType dumb and the corresponding combinator smart

-export class InterfaceType<P extends Props> extends Type<any, InterfaceOf<P>> {
+export class InterfaceType<P extends Props, A> extends Type<any, A> {

/** @alias `interface` */
-export const type = <P extends Props>(props: P, name: string = getNameFromProps(props)): InterfaceType<P> =>
+export const type = <P extends Props>(
+  props: P,
+  name: string = getNameFromProps(props)
+): InterfaceType<P, InterfaceOf<P>> =>  

I put up a 95 branch where all the runtime type classes are dumb. I tested it against the following scenario, seems to work fine

import * as t from 'io-ts'

type GenerableProps = { [key: string]: Generable }
type GenerableInterface = t.InterfaceType<GenerableProps, any>
type GenerableStrict = t.StrictType<GenerableProps, any>
type GenerablePartials = t.PartialType<GenerableProps, any>
interface GenerableDictionary extends t.DictionaryType<Generable, Generable, any> {}
interface GenerableRefinement extends t.RefinementType<Generable, any, any> {}
interface GenerableArray extends t.ArrayType<Generable, any> {}
interface GenerableUnion extends t.UnionType<Array<Generable>, any> {}
interface GenerableIntersection extends t.IntersectionType<Array<Generable>, any> {}
interface GenerableTuple extends t.TupleType<Array<Generable>, any> {}
interface GenerableReadonly extends t.ReadonlyType<Generable, any> {}
interface GenerableReadonlyArray extends t.ReadonlyArrayType<Generable, any> {}
type Generable =
  | t.StringType
  | t.NumberType
  | t.BooleanType
  | GenerableInterface
  | GenerableRefinement
  | GenerableArray
  | GenerableStrict
  | GenerablePartials
  | GenerableDictionary
  | GenerableUnion
  | GenerableIntersection
  | GenerableTuple
  | GenerableReadonly
  | GenerableReadonlyArray
  | t.LiteralType<any>
  | t.KeyofType<any>

function f(generable: Generable): string {
  switch (generable._tag) {
    case 'InterfaceType':
      return Object.keys(generable.props)
        .map(k => f(generable.props[k]))
        .join('/')
    case 'StringType':
      return 'StringType'
    case 'NumberType':
      return 'StringType'
    case 'BooleanType':
      return 'BooleanType'
    case 'RefinementType':
      return f(generable.type)
    case 'ArrayType':
      return 'ArrayType'
    case 'StrictType':
      return 'StrictType'
    case 'PartialType':
      return 'PartialType'
    case 'DictionaryType':
      return 'DictionaryType'
    case 'UnionType':
      return 'UnionType'
    case 'IntersectionType':
      return 'IntersectionType'
    case 'TupleType':
      return generable.types.map(type => f(type)).join('/')
    case 'ReadonlyType':
      return 'ReadonlyType'
    case 'ReadonlyArrayType':
      return 'ReadonlyArrayType'
    case 'LiteralType':
      return 'LiteralType'
    case 'KeyofType':
      return 'KeyofType'
  }
}

const schema = t.interface({
  a: t.string,
  b: t.union([
    t.partial({
      c: t.string,
      d: t.literal('eee')
    }),
    t.boolean
  ]),
  e: t.intersection([
    t.interface({
      f: t.array(t.string)
    }),
    t.interface({
      g: t.union([t.literal('toto'), t.literal('tata')])
    })
  ])
})

f(schema) // OK!

@sledorze could you please try out this possible fix in your use case?

[gcanti]

Forgot to handle recursive types

import * as t from 'io-ts'

type GenerableProps = { [key: string]: Generable }
type GenerableInterface = t.InterfaceType<GenerableProps, any>
type GenerableStrict = t.StrictType<GenerableProps, any>
type GenerablePartials = t.PartialType<GenerableProps, any>
interface GenerableDictionary extends t.DictionaryType<Generable, Generable, any> {}
interface GenerableRefinement extends t.RefinementType<Generable, any, any> {}
interface GenerableArray extends t.ArrayType<Generable, any> {}
interface GenerableUnion extends t.UnionType<Array<Generable>, any> {}
interface GenerableIntersection extends t.IntersectionType<Array<Generable>, any> {}
interface GenerableTuple extends t.TupleType<Array<Generable>, any> {}
interface GenerableReadonly extends t.ReadonlyType<Generable, any> {}
interface GenerableReadonlyArray extends t.ReadonlyArrayType<Generable, any> {}
interface GenerableRecursive extends t.RecursiveType<Generable, any> {}
type Generable =
  | t.StringType
  | t.NumberType
  | t.BooleanType
  | GenerableInterface
  | GenerableRefinement
  | GenerableArray
  | GenerableStrict
  | GenerablePartials
  | GenerableDictionary
  | GenerableUnion
  | GenerableIntersection
  | GenerableTuple
  | GenerableReadonly
  | GenerableReadonlyArray
  | t.LiteralType<any>
  | t.KeyofType<any>
  | GenerableRecursive
  | t.UndefinedType

function f(generable: Generable): string {
  switch (generable._tag) {
    case 'InterfaceType':
      return Object.keys(generable.props)
        .map(k => f(generable.props[k]))
        .join('/')
    case 'StringType':
      return 'StringType'
    case 'NumberType':
      return 'StringType'
    case 'BooleanType':
      return 'BooleanType'
    case 'RefinementType':
      return f(generable.type)
    case 'ArrayType':
      return 'ArrayType'
    case 'StrictType':
      return 'StrictType'
    case 'PartialType':
      return 'PartialType'
    case 'DictionaryType':
      return 'DictionaryType'
    case 'UnionType':
      return 'UnionType'
    case 'IntersectionType':
      return 'IntersectionType'
    case 'TupleType':
      return generable.types.map(type => f(type)).join('/')
    case 'ReadonlyType':
      return 'ReadonlyType'
    case 'ReadonlyArrayType':
      return 'ReadonlyArrayType'
    case 'LiteralType':
      return 'LiteralType'
    case 'KeyofType':
      return 'KeyofType'
    case 'RecursiveType':
      return f(generable.type)
    case 'UndefinedType':
      return 'UndefinedType'
  }
}

const schema = t.interface({
  a: t.string,
  b: t.union([
    t.partial({
      c: t.string,
      d: t.literal('eee')
    }),
    t.boolean
  ]),
  e: t.intersection([
    t.interface({
      f: t.array(t.string)
    }),
    t.interface({
      g: t.union([t.literal('toto'), t.literal('tata')])
    })
  ])
})

f(schema) // OK!

type Rec = {
  a: number
  b: Rec | undefined
}

const Rec = t.recursion<Rec, Generable>('T', self =>
  t.interface({
    a: t.number,
    b: t.union([self, t.undefined])
  })
)

f(Rec) // OK!

[sledorze]

@gcanti Ok thanks, I'm on it!

[sledorze]

@gcanti WORKS!!!

Awesome!!!

https://gist.github.com/sledorze/57c45b27aa57d2971227ba5ac5f41edf

And it correctly errors when try to pass incorrect structures (encoded by the two TObjType and TType unions)

[gcanti]

Ok, going to amend Flow's definition files. I'll send a PR for reviews.

/cc @hallettj

[sledorze]

would be awesome to have a 'next' version to test against the rest of the code base.

[sledorze]

@gcanti just asking what is the expected timing on this (need to do some development that could be done in a totally different way if having that and pondering if I should delay it or not).
Not pressure, or commitment requested; I ask purely for an informative point of view..
/cc @hallettj

[gcanti]

@sledorze For what concerns TypeScript I'm sold on this change. Actually, since being able to do such type-safe introspections is an essential feature of io-ts, I consider the current behaviour of 0.9.0 a bug, so I'm going to merge https://github.com/gcanti/io-ts/pull/96 a release a patch. @hallettj please let me know if you find some inconsistencies in the Flow definition files, we'll amend them in a later release.

[gcanti]

Closing via https://github.com/gcanti/io-ts/releases/tag/0.9.1