Io-ts: High-level explanation of API changes

[mmkal]

📖 Documentation

Would it be possible to add a paragraph or two to the readme, on the differences between "experimental" (Decoder/Encoder/Codec/Schema) and "stable" (Type), and who should use which? Assuming the concept of Type will be removed in v3, does it mean usage like

import * as t from 'io-ts'

const Person = t.type({ name: t.string })

Person.decode({ name: 'Alice' })

Will need to change? If so, will there be a migration guide (I assume Codec largely replaces Type)? I've seen the S.make(S => ...) syntax around in a few places, but it's not immediately clear if applications relying on io-ts will need to use it, or if it's mainly a low-level construct.

A link to a tracking issue could also work.

[gcanti]

Decoder, Encoder and Codec modules are published as experimental in order to get early feedback from the community.

Codec should largely replace Type except for untagged unions (that's because encoders don't support them).

The Schema module is an advanced feature which must prove its worth, the goal would be to express a generic schema and then derive from that multiple concrete instances (like decoders, encoders, equality instances, arbitraries, etc...).

I think that speaking about a v3 is too early, the new modules / APIs must be validated first.

I'm reserving the 2.2 label to track the issues related to the new modules.

For what concerns the APIs changes, here's a tiny migration guide in order to help experimenting with the Decoder module:

• Decoder is defined with only one type parameter
• keyof is replaced by literal
• record doesn't accept a domain schema anymore
• brand is replaced by refinement / parse which are not opinionated on how you want to define your branded types
• recursive renamed to lazy (mutually recursive decoders are supported)
• intersect is now pipeeable
• tagged unions must be explicitly defined (with sum) in order to get optimized
• tuple is not limited to max five components

[mmkal]

Thanks - some follow-up questions:

Codec should largely replace Type except for untagged unions (that's because encoders don't support them).

Will untagged unions be supported at all?

• Decoder is defined with only one type parameter

Does that mean there's no longer such a thing as a Decoder from I to A - effectively I is always unknown?

• keyof is replaced by literal

Will this affect the advice to use t.keyof({ a: null, b: null }) to achieve enum-like behaviour (potentially related to the answer to "Will untagged unions be supported at all")?

• brand is replaced by refinement / parse which are not opinionated on how you want to define your branded types

Does that mean anything for https://github.com/gcanti/io-ts/issues/373? The request there was to keep un-branded refinements as an option.

[steida]

Will this be possible as Codec? I mean, the possibility to decode from two different types.

export const FaunaDocRef = (() => {
  const Serialized = t.type({
    '@ref': t.type({
      id: FaunaID,
      collection: t.type({
        '@ref': t.type({
          id: t.string,
          collection: t.type({
            '@ref': t.type({ id: t.literal('collections') }),
          }),
        }),
      }),
    }),
  });
  const FaunaDocRef = t.type({
    id: FaunaID,
    collection: t.string,
  });
  type FaunaDocRef = t.TypeOf<typeof FaunaDocRef>;
  return new t.Type<FaunaDocRef, values.Ref, unknown>(
    'FaunaDocRef',
    FaunaDocRef.is,
    (u, c) => {
      if (u instanceof values.Ref) {
        return u.collection
          ? t.success({
              id: u.id as FaunaID, // as FaunaID is ok, we don't create ids anyway
              collection: u.collection.id,
            })
          : t.failure(u, c);
      }
      return either.either.chain(Serialized.validate(u, c), (s) =>
        t.success({
          id: s['@ref'].id,
          collection: s['@ref'].collection['@ref'].id,
        }),
      );
    },
    (a) =>
      new values.Ref(
        a.id,
        new values.Ref(a.collection, values.Native.COLLECTIONS),
      ),
  );
})();
export type FaunaDocRef = t.TypeOf<typeof FaunaDocRef>;

[gcanti]

Will untagged unions be supported at all?

They are supported in Decoder. I can't find a reliable way to support them in Encoder though.

So either you define an encoder by hand...

import { left, right } from 'fp-ts/lib/Either'
import * as C from 'io-ts/lib/Codec'
import * as D from 'io-ts/lib/Decoder'
import * as G from 'io-ts/lib/Guard'

const NumberFromString: C.Codec<number> = C.make(
  D.parse(D.string, (s) => {
    const n = parseFloat(s)
    return isNaN(n) ? left(`cannot decode ${JSON.stringify(s)}, should be NumberFromString`) : right(n)
  }),
  { encode: String }
)

export const MyUnion: C.Codec<number | string> = C.make(D.union(NumberFromString, D.string), {
  encode: (a) => (G.string.is(a) ? a : NumberFromString.encode(a))
})

...or you extend Codec to something containing a guard

import * as E from 'io-ts/lib/Encoder'

interface Compat<A> extends D.Decoder<A>, E.Encoder<A>, G.Guard<A> {}

function make<A>(codec: C.Codec<A>, guard: G.Guard<A>): Compat<A> {
  return {
    is: guard.is,
    decode: codec.decode,
    encode: codec.encode
  }
}

function union<A extends ReadonlyArray<unknown>>(
  ...members: { [K in keyof A]: Compat<A[K]> }
): Compat<A[number]> {
  return {
    is: G.guard.union(...members).is,
    decode: D.decoder.union(...members).decode,
    encode: (a) => {
      for (const member of members) {
        if (member.is(a)) {
          return member.encode(a)
        }
      }
    }
  }
}

const string = make(C.string, G.string)

const NumberFromString2 = make(NumberFromString, G.number)

export const MyUnion2: Compat<number | string> = union(NumberFromString2, string)

..or... something else? I don't know, any idea?

Does that mean there's no longer such a thing as a Decoder from I to A - effectively I is always unknown?

Yes it is, but please note that basically I is always unknown in the stable API too.

Will this affect the advice to use t.keyof({ a: null, b: null }) to achieve enum-like behaviour

Not sure what you mean, but the new way is

import * as D from 'io-ts/lib/Decoder'

const MyEnum = D.literal('a', 'b')

and literal is supported by Encoder, so you don't need untagged unions for that.

Does that mean anything for #373? The request there was to keep un-branded refinements as an option

The new refinement function has the following signature

export declare function refinement<A, B extends A>(
  from: Decoder<A>,
  refinement: (a: A) => a is B,
  expected: string
): Decoder<B>

where B is supposed to be different from A, while the old refinement function has the following signature

export declare function refinement<C extends Any>(
  codec: C,
  predicate: Predicate<TypeOf<C>>,
  name?: string
): RefinementC<C>

which I still consider a bad API since the predicate is not carried to the type level.

[steida]

@gcanti Will be JSON type possible?

[IMax153]

@steida Here is how I solved it using the suggestions from @gcanti above.

import * as C from 'io-ts/lib/Codec';
import * as D from 'io-ts/lib/Decoder';
import * as E from 'io-ts/lib/Encoder';
import * as G from 'io-ts/lib/Guard';

export interface Compat<A> extends D.Decoder<A>, E.Encoder<A>, G.Guard<A> {}

export const makeCompat: <A>(c: C.Codec<A>, g: G.Guard<A>) => Compat<A> = (c, g) => ({
  is: g.is,
  decode: c.decode,
  encode: c.encode,
});

export const lazy = <A>(id: string, f: () => Compat<A>): Compat<A> => {
  return makeCompat(C.lazy(id, f), G.guard.lazy(id, f));
};

export const untaggedUnion: <A extends ReadonlyArray<unknown>>(
  ...ms: { [K in keyof A]: Compat<A[K]> }
) => Compat<A[number]> = (...ms) => ({
  is: G.guard.union(...ms).is,
  decode: D.decoder.union(...ms).decode,
  encode: (a) => ms.find((m) => m.is(a)),
});

type Json = string | number | boolean | null | { [property: string]: Json } | Json[];

const Json: Compat<Json> = lazy<Json>('Json', () =>
  untaggedUnion(
    makeCompat(C.string, G.string),
    makeCompat(C.number, G.number),
    makeCompat(C.boolean, G.boolean),
    makeCompat(C.literal(null), G.literal(null)),
    makeCompat(C.record(Json), G.record(Json)),
    makeCompat(C.array(Json), G.array(Json)),
  ),
);

const json = Json.decode([1, [[['1']], { a: 1, b: false }]]);
console.log(json);
// {
//    _tag: 'Right',
//    right: [ 1, [ [ [ '1' ] ], { a: 1, b: false } ] ]
// }

[gcanti]

@IMax153 @steida the Json encoder is just the identity function

import * as C from 'io-ts/lib/Codec'
import * as D from 'io-ts/lib/Decoder'
import * as E from 'io-ts/lib/Encoder'

type Json = string | number | boolean | null | { [key: string]: Json } | Array<Json>

const JsonDecoder = D.lazy<Json>('Json', () =>
  D.union(C.string, C.number, C.boolean, C.literal(null), C.record(Json), C.array(Json))
)

const Json: C.Codec<Json> = C.make(JsonDecoder, E.id)

[mmkal]

..or... something else? I don't know, any idea?

@gcanti maybe there could be a special case for unions of codecs with encode = t.identity. If t.identity itself were branded, then you could know at the type level and at runtime that it's safe to encode with any of the sub-types. It would cover the "simple" cases where io-ts is just used for validation, which are quite common:

t.union([t.string, t.number, t.type({ myProp: t.boolean })])

☝️ that's assuming it's possible to "propagate" the identity encoder from props, i.e. t.type({ myProp: t.boolean }) satisfies the requirement of all its props being identity-encoders, so it is one too.

note that basically I is always unknown in the stable API too.

Not in the case of typeA.pipe(typeB), but it sounds like that whole flow is going to change somewhat - if so, some kind of migration might be worth noting somewhere in the docs.

[gcanti]

maybe there could be a special case for unions of codecs with encode = t.identity

@mmkal maybe, but supporting untagged unions means that the implementation of Schemable's union

declare function union<A extends ReadonlyArray<unknown>>(
  ...members: { [K in keyof A]: Encoder<A[K]> }
): Encoder<A[number]>

should work with any Encoder.

Not in the case of typeA.pipe(typeB)

I think that most of the times .pipe(...) can be replaced by parse

declare function parse<A, B>(from: Decoder<A>, parser: (a: A) => Either<string, B>): Decoder<B>

[leemhenson]

I think it would be useful to include a pipe-like fn anyway, for when you already have two decoders that you want to sequence:

const pipe = <A>(da: D.Decoder<A>) => <B>(db: D.Decoder<B>): D.Decoder<B> => ({
  decode: flow(da.decode, E.chain(db.decode)),
});

Or, I dunno, would calling that D.chain make more sense? That's what I instinctively reached for.

[leemhenson]

Another thing I miss from the new API is a name attribute on a Decoder/Encoder. I like to wrap the library's own DecodeError in my own Error subclass so I can attach more metadata, using something like this:

export const decode = <A>(
  decoder: Decoder<A>,
  errorMetadata?: Record<string, unknown>
) =>
  flow(
    decoder.decode,
    E.mapLeft(error => makeIoTsDecodeError(error, errorMetadata))
  );

I used to be able to automatically smoosh in the Type<A, O, I>.name:

makeIoTsDecodeError(error, { decoderName: decoder.name, ...errorMetadata }

But that's not possible with Decoder<A> now.

I suppose I could extend Decoder to add it back in just inside my projects but I wonder what was the reason for dropping name?

[gcanti]

I think it would be useful to include a pipe-like fn anyway, for when you already have two decoders that you want to sequence

example?

I wonder what was the reason for dropping name?

Because it makes the APIs unnecessarily complicated, IMO the error messages are readable even without the names (and you can always use withExpected if you don't like the default)

[leemhenson]

example?

I know D.parse exists for effectively chaining A => B on the end of an earlier decoder, but you have to return Either<string, B>. If I already have a decoder that provides A => B then it's easiest for me to just compose them together. I suppose there's some tension there because decode is always taking u: unknown as opposed to a hard requirement on A, which D.parse does give you. 🤷

In some ways this feels similar to the discussion we had recently about re-adding the second type parameter to Encoder. It almost feels to me like we should have Decoder<Output, Input = unknown>, and D.chain(dua: Decoder<A>, dab: Decoder<B, A>) => Decoder<B>.

withExpected

Yeah I think i need to experiment with that a bit - I guess I would use it to replace the text inside a leaf?

[mmkal]

@gcanti re D.parse vs the old Type.prototype.pipe. What would you recommend for a base64-json decoder. i.e. one that decodes base 64, parses the decoded string as JSON, then validates the json using io-ts. An example use case is handling kinesis events, which trigger lambdas with base64 payloads. With the current io-ts, it's possible to use a combinator like:

const MyEvent = kinesisEvent(t.type({ foo: t.string }))

The kinesisEvent combinator can ensure:

• the input value looks like { records: Array<{ recordId: string; data: string }> }
• each records[*].data is a string
• each string is valid base 64
• when decoded, the strings are valid json
• when json-parsed, the decoded strings have structure { foo: string }

Is this possible with the new D.parse? Would you recommend @leemhenson's method - if so it would be great if there were a first-class helper for it to avoid many implementations in downstream projects that might miss edge cases.

---

Another question, since this issue title is "High-level explanation of API changes". Could you give a recommendation for users who rely on myType.props for interface and partial types, and myType.types for union and intersection types in the stable API? From comments like this it sounds like there isn't a replacement yet, for reflection-like functionality. Use cases include UI-generation, dynamic codec manipulation, etc. In this comment you mentioned development of the stable API is frozen. Does this mean the functionality is going to be replaced by something else?

[gcanti]

Is this possible with the new D.parse?

Yes, but there are many different ways to get the final result so I guess it really depends on your coding style ("everything is a decoder" or "I want to lift by business / parsing logic only once"?).

For example

import * as E from 'fp-ts/lib/Either'
import { Json } from 'io-ts/lib/JsonEncoder'
import * as D from 'io-ts/lib/Decoder'
import { flow } from 'fp-ts/lib/function'

// > decodes base 64
declare function decodeBase64(s: string): E.Either<string, string>

// > parses the decoded string as JSON
declare function parseJSON(s: string): E.Either<string, Json>

// > then validates the json using io-ts
declare function decodeItem(json: Json): E.Either<string, { foo: string }>

const parser = flow(decodeBase64, E.chain(parseJSON), E.chain(decodeItem))

export const X = D.parse(D.string, parser)

it sounds like there isn't a replacement yet

Actually one of the goals of my rewrite was to get rid of those meta infos (at the type level)

[gcanti]

@leemhenson @mmkal given the good results in https://github.com/gcanti/io-ts/issues/478 I'm going to make the following breaking changes:

• Decoder
  
  
  
  • change DecoderError
  
  
  • remove never
  
  
  • make parse pipeable and change its parser argument
  
  
• Guard
  
  
  
  • remove never
  
  
• Schemable
  
  
  
  • make intersections pipeables
  
  
  • make refinements pipeables
  
  

parse

from

declare export function parse<A, B>(from: Decoder<A>, parser: (a: A) => Either<string, B>): Decoder<B>

to

declare export function parse<A, B>(parser: (a: A) => E.Either<DecodeError, B>): (from: Decoder<A>) => Decoder<B>

Pros:

• more general (DecodeError instead of string)
• pipeable
• should accomodate the pipe use case

import { pipe } from 'fp-ts/lib/pipeable'
import * as D from '../src/Decoder2'
import { Json } from '../src/JsonEncoder'

// > decodes base 64
declare const Base64: D.Decoder<string>

// > parses the decoded string as JSON
declare const Json: D.Decoder<Json>

// > then validates the json using io-ts
declare const Item: D.Decoder<{ foo: string }>

export const X = pipe(D.string, D.parse(Base64.decode), D.parse(Json.decode), D.parse(Item.decode))

[leemhenson]

Are you changing the signatue of

export interface Decoder<A> {
  readonly decode: (u: unknown) => Either<DecodeError, A>
}

to:

export interface Decoder<A, B = unknown> {
  readonly decode: (u: B) => Either<DecodeError, A>
}

?

Otherwise Base64 and Json are both going to have to repetitively test inside their decode implementations whether u is actually a string before performing string-based operations. Micro-optimizations maybe, but as you combine more and more parsers together to operate on larger and larger structures, it could start to add up.

[gcanti]

@leemhenson isn't what happens in your proposal too?

const pipe = <A>(da: D.Decoder<A>) => <B>(db: D.Decoder<B>): D.Decoder<B> => ({
  decode: flow(da.decode, E.chain(db.decode)),
});

[leemhenson]

Why is that in bold? Emphasis not mine! 😬

[gcanti]

@leemhenson maybe it's just a bias of mine but I consider a "proper decoder" an arrow that goes from unknown to some type A

unknown -> M<A>

for some effect M, because otherwise it's just a "normal" kleisli arrow

A -> M<B>

and I use chain to compose those.

So personally I would model my pipeline starting from normal kleisli arrows and then I would define a suitable decoder based on the use case at hand.

// kleisli arrows in my domain

declare function decodeBase64(s: string): E.Either<string, string>

declare function parseJSON(s: string): E.Either<string, Json>

declare function decodeItem(json: Json): E.Either<string, { foo: string }>

// I can compose them as usual using the `Monad` instance of `Either`

const decode = flow(decodeBase64, E.chain(parseJSON), E.chain(decodeItem))

// and then define my decoder

export const MyDecoder = pipe(
  D.string,
  D.parse((s) =>
    pipe(
      decode(s),
      E.mapLeft((e) => D.error(s, e))
    )
  )
)

Alternatively I could have already defined some decoders

declare const Base64: D.Decoder<string>

declare const Json: D.Decoder<Json>

declare const Item: D.Decoder<{ foo: string }>

if this is the case, again I can compose them via parse

// and I can compose them via parse

export const MyDecoder2 = pipe(Base64, D.parse(Json.decode), D.parse(Item.decode))

// or even this if I want micro optimizations
export const MyDecoder3 = pipe(
  Base64,
  D.parse((s) =>
    pipe(
      parseJSON(s),
      E.mapLeft((e) => D.error(s, e))
    )
  ),
  D.parse(Item.decode)
)

Do we really need something more? Genuine question, I'm open to suggestions if you think there's an ergonomic issue with the current APIs.

In the end if you can define a

export interface Decoder<A, B> {
  readonly decode: (a: A) => Either<DecodeError, B>
}

then you can just define f = (a: A) => Either<DecodeError, B> and use parse to compose f with a Decoder<A>

[leemhenson]

It's not a major issue, just a niggle I keep encountering because I have scenarios like these:

• desire to decode from some wire representation into a branded type, for simplicity let's say Int
• wire types might be numeric or a string representation of a number, e.g. 3 or "3"
• an Int might also be further branded into FooId or Cents or something, and that logic might include doing some bounds checking

So in this case I would like to have primitive decoders:

intFromNumber: Decoder<Int, number>
intFromString: Decoder<Int, string>
fooIdFromInt: Decoder<FooId, Int>
centsFromInt: Decoder<Cents, Int>

then I can compose them together to make complex ones:

pipe(
  stringFromUnknown,
  intFromString,
  fooIdFromInt,
) // => Decoder<FooId, unknown>

I could wrap all that logic up using Decoder<FooId> as it is today but if I have useful chains of decoders that I want to reuse it always ends up feeling like it would be more elegant to make the composition at the Decoder level.

But, hey, that's just me. I might just be using the wrong tool for the job. 🤷

[gcanti]

it always ends up feeling like it would be more elegant to make the composition at the Decoder level

Well, while I love io-ts, as a user I would try (as far as possible) to not leak an implementation detail (i.e. which library I'm using to validate / decode at the border). In my app domain I would prefer to define

intFromNumber: number -> Either<MyDomainError, Int>
intFromString: string -> Either<MyDomainError, Int>
fooIdFromInt: Int -> Either<MyDomainError, FooId>
centsFromInt: Int -> Either<MyDomainError, Cents>

that will be future-proof even if a I replace io-ts with another solution.

In elm-ts I removed the hard dependency on io-ts by just requiring a kleisli arrow.
I will do the same for fp-ts-routing in the next major release.

But that's just a point of view, yours is sensible too.

Let me just think more about all of this...

[leemhenson]

... as a user I would try (as far as possible) to not leak an implementation detail (i.e. which library I'm using to validate / decode at the border). In my app domain I would prefer to define ...

Yeah I'm only talking about composition of Decoders as a means to construct the larger Decoders that I use at the app boundary to convert unknown => Either<DecodeError, SomeComplexNestedProduct>. I don't see the Decoder in the rest of the codebase.

[gcanti]

In the end if you can define a

export interface Decoder {
readonly decode: (a: A) => Either
}

then you can just define f = (a: A) => Either

However the converse is also true, so my POV is actually biased and without noticeable substance, @leemhenson I'll reconsider my position.

[gcanti]

@leemhenson while experimenting with kleisli arrows looks like I found something more general than DecoderT (see the Kleisli module):

interface Kleisli<M extends URIS2, I, E, A> {
  readonly decode: (i: I) => Kind2<M, E, A>
}

for which I can define a compose operation

const compose = <M extends URIS2, E>(M: Monad2C<M, E>) => <A, B>(ab: Kleisli<M, A, E, B>) => <I>(
  ia: Kleisli<M, I, E, A>
): Kleisli<M, I, E, B> => ({
  decode: (i) => M.chain(ia.decode(i), ab.decode)
})

Then from Kleisli I can derive KleisliDecoder

interface KleisliDecoder<I, A> extends K.Kleisli<E.URI, I, DecodeError, A> {}`

and from KleisliDecoder I can derive our old Decoder

interface Decoder<A> extends KD.KleisliDecoder<unknown, A> {}

Example

import { pipe } from 'fp-ts/lib/pipeable'
import * as D from '../src/Decoder'
import * as KD from '../src/KleisliDecoder'

interface IntBrand {
  readonly Int: unique symbol
}
type Int = number & IntBrand
interface CentsBrand {
  readonly Cents: unique symbol
}
type Cents = number & CentsBrand

declare const IntFromString: KD.KleisliDecoder<string, Int>
declare const CentsFromInt: KD.KleisliDecoder<Int, Cents>

// const result: D.Decoder<Cents>
export const result = pipe(
  D.string, 
  D.compose(IntFromString), 
  D.compose(CentsFromInt)
)

[leemhenson]

Love it.

[gcanti]

KleisliDecoder is quite interesting in that its input type is fine grained (i.e. not a generic Record<string, I>) and depends on the fields passed in

/*
const kdecoder: KD.KleisliDecoder<{
    name: unknown;
    age: string;
    cents: Int;
}, {
    name: string;
    age: Int;
    cents: Cents;
}>
*/
export const kdecoder = KD.type({
  name: D.string,
  age: IntFromString,
  cents: CentsFromInt
})

EDIT: same for tuple, etc...

// const kdecoder2: KD.KleisliDecoder<[unknown, string, Int], [string, Int, Cents]>
export const kdecoder2 = KD.tuple(D.string, IntFromString, CentsFromInt)

[gcanti]

v2.2.7 released https://github.com/gcanti/io-ts/releases/tag/2.2.7

[osdiab]

as someone not super well versed in category theory, when does it make sense to use KleisliDecoder functions? I get the general concept of Kleisli arrows and can figure it out when it gets stable but I predict difficulty explaining these things to my team lol

is it mostly that Decoder functions assume unknown as the input type/are generally implemented in terms of the KleisliDecoder functions, but KleisliDecoder functions also allow control over the input type (in a sense closer to the original io-ts Type generic params)? and how does that relate to Kleisli arrows? sorry for the question but i'm kinda out of my depth

[gcanti]

but KleisliDecoder functions also allow control over the input type (in a sense closer to the original io-ts Type generic params)

@osdiab that's right.

This is the definition of Type in index.ts

interface Decoder<I, A> {
  readonly decode: (i: I) => Either<Errors, A>
  ..other fields..
}

class Type<A, O = A, I = unknown> implements Decoder<I, A>, Encoder<A, O> { ... }

so KleisliDecoder<I, A> is pretty similar to the old Decoder<I, A> (but without the other fields)

interface KleisliDecoder<I, A> {
  readonly decode: (i: I) => Either<DecodeError, A>
}

and how does that relate to Kleisli arrows?

In general a "Kleisli arrow" is a function with the following signature:

I -> M<A>

where M is a data type. So a KleisliDecoder<I, A> can be considered a Kleisli arrow:

I -> Either<E, A>

where M = Either<E, ?> is the Either data type.

[mmkal]

This is very helpful @gcanti - thank you! A couple of questions about KleisliDecoder:

1. Might it be worth renaming Decoder -> UnknownDecoder and KleisliDecoder -> Decoder, since that's the naming scheme consumers are used to already? (plus, it avoids intimidating category theory language for a core type). Similarly to how Encoders were renamed to be more broad

2. Would you consider an equivalent for Codecs? A nice feature to go along with

// const result: D.Decoder<Cents>
export const result = pipe(
  D.string, 
  D.compose(IntFromString), 
  D.compose(CentsFromInt)
)

would be the ability to go "the other way". In the case of the above, there may be no need since all IntFromString and CentsFromInt do is add brands so don't affect serialisation. But going back to the string -> base64-decode -> json-parse -> decode use-case, if we do have Codecs for each step, being able to compose them together would be valuable.

example (note - this likely doesn't compile with the experimental API, I'm just trying to give an idea, mostly based on how the stable Type interface works)

declare const base64: Codec<string, string>
declare const json: Codec<Json, string>
const user = C.type({ name: C.string, dob: C.DateFromISOString })

// const result: C.Codec<string, { name: string; dob: Date }>
const result = pipe(C.string, C.compose(base64), C.compose(json), C.compose(user))

Then result.decode would take a base64 string, decode it, parse it as json, then validate it matches the user interface, as well as parse the dob property as an ISO string into a Date object. A failure at each step would get an appropriate error message, just like with the composed KleisliDecoders. result.encode would do the opposite: stringify the date, json stringify, and base64-encode.

With an example value:

const bob = result.decode('eyJuYW1lIjoiQm9iIiwiZG9iIjoiMTk4MC0wMS0wMVQwMDowMDowMC4wMDBaIn0=')
expect(bob.right).toEqual({ name: 'Bob', dob: new Date('1980') })

const str = result.encode(bob.right)
expect(str).toEqual('eyJuYW1lIjoiQm9iIiwiZG9iIjoiMTk4MC0wMS0wMVQwMDowMDowMC4wMDBaIn0=')

[osdiab]

I'm also somewhat confused why the Decoder type wouldn't also be a KleisliDecoder, because it also decodes to an Either (and in fact many are implemented directly as aliases of KleisliDecoder functions) - just the parameter is explicitly unknown, ie unknown -> Either<E, A>

[gcanti]

Might it be worth renaming Decoder -> UnknownDecoder and KleisliDecoder -> Decoder, since that's the naming scheme consumers are used to already? (plus, it avoids intimidating category theory language for a core type)

@mmkal I agree. I'd go even further, I'd merge those two modules into one if possible (working on a POC)

[gcanti]

@mmkal I got something to show (branch 453)

Here's the main changes:

• make Codec more general by adding a I type parameter
• make Decoder more general by adding a I type parameter
  
  
  
  • remove KleisliDecoder module
  
  
• make TaskDecoder more general by adding a I type parameter
  
  
  
  • remove KleisliTaskDecoder module
  
  
• make Guard more general by adding a I type parameter

I'd merge those two modules into one

Done.

Would you consider an equivalent for Codecs?

Here's the test you are more interested in.

Complete changelog

[steida]

@gcanti Am I correct that experimental schema replaces io-ts stable "reflection" like props, types etc.? I would like to write codec/schema which can be used to build UI. Or should I create io-ts codecs from separate JSON data instead?

const Direction = t.union([t.literal('left'), t.literal('right')]);
// Here I can extract types...
Direction.types

[gcanti]

Am I correct that experimental schema replaces io-ts stable "reflection" like props, types etc.?

@steida not exactly, the purpose of Schema is to be a compilation source, then you choose the compilation target by picking a suitable Schemable instance (Decoder, Eq, etc...).

So let's say this is the model of your form

import * as D from 'io-ts/lib/Decoder'

interface Form<A> {
  readonly decoder: D.Decoder<A>
  readonly render: (onValue: (a: A) => void) => JSX.Element
}

If you can define a Schemable instance for it...

import { Schemable1 } from 'io-ts/lib/Schemable'

declare module 'fp-ts/lib/HKT' {
  interface URItoKind<A> {
    readonly Form: Form<A>
  }
}

declare const schemableForm: Schemable1<'Form'>

...then you could use Schema to define a source and schemableForm to compile

import * as S from 'io-ts/lib/Schema'

const Person = S.make((S) =>
  S.type({
    name: S.string,
    age: S.number
  })
)

/*
const myform: Form<{
    name: string;
    age: number;
}>
*/
const myform = S.interpreter(schemableForm)(Person)

[steida]

@gcanti Thank you. So this is the only replacement of props, types of io-ts stable?

[kpritam]

Hi @gcanti , We are starting new project and planning to use io-ts.

Looking at this issue, what do you suggest? Should we use stable Type module or go with new Codec module (especially Decoder)?

Note: We are mostly interested in decoding json coming from server side.

[gcanti]

@mmkal v2.2.8 released

We are mostly interested in decoding json coming from server side

@kpritam I don't expect drastic changes in the Decoder module but in all honesty I can't give you a 100% guarantee.

Surely there are APIs that I consider more "stable" than others, mainly to make the migration from the old Type easier, namely:

• string
• number
• boolean
• UnknownArray
• UnknownRecord
• literal
• type
• partial
• array
• record
• tuple
• union
• sum
• intersect
• refine
• parse
• lazy
• TypeOf
• failure
• success

[osdiab]

another thought on my part: i was trying to contribute to io-ts-reporters for the issue https://github.com/gillchristian/io-ts-reporters/issues/38 (make the reporter show the name of the codec when printing errors) but for the old codecs it seems that the actual expected type is stored in error.context.type.name and gets erased if you provide a name; and the new codecs only take a decoder and encoder so there's no obvious way to give these codecs a name.

knowing which codec failed makes debugging codec decode errors easier for a large project in my limited time using io-ts, and having some canonical name for a codec is a decent way to do that - any chance there might be a way to still give these codecs an optional name (like how withMessage works now) without losing the expected type information, and preferably show up in the default error reporter's output? Considered withMessage but if some parent type decoding fails I don't particularly want a child type's message to get overwritten either.

[gcanti]

@osdiab here's a possible withMessage implementation

import * as E from 'fp-ts/lib/Either'
import { pipe } from 'fp-ts/lib/function'
import * as DE from 'io-ts/lib/DecodeError'
import * as D from 'io-ts/lib/Decoder'
import * as FS from 'io-ts/lib/FreeSemigroup'

const withMessage = (message: string): (<I, A>(decoder: D.Decoder<I, A>) => D.Decoder<I, A>) =>
  D.mapLeftWithInput((_, e) => FS.of(DE.lazy(message, e)))

const Person = pipe(
  D.type({
    name: D.string,
    age: D.number
  }),
  withMessage('Person')
)

console.log(
  pipe(
    Person.decode({}),
    E.fold(D.draw, () => 'no errors')
  )
)
/*
lazy type Person
├─ required property "name"
│  └─ cannot decode undefined, should be string
└─ required property "age"
   └─ cannot decode undefined, should be number
*/

(the output message looks a bit weird because I'm piggybacking the existing DE.lazy decode error, but we could add a another specialized error type for this use case)

[williamareynolds]

This might be the best place to bring this up, but I'm happy to create an issue (and PR if I'm correct) if desired. When using the current latest fp-ts version 2.8.1 I find that the schema docs seem to be incorrect.

After following the instructions, I see a type error on my custom implementation of the Decoder schemable:

After a bit of digging, I found that in the lib, Decoder's schemable is actually of the type Schemable2C. By creating my own extension of Schemable2C, an additional overload of interpreter, and using the Schemable2C type for my custom decoder, the type error goes away. I'm happy to write a PR to add this info to the docs if this is the correct solution.

[gcanti]

@williamareynolds I totally forgot to update Schema.md after the last release, thanks for pointing out (should be ok now)

[osdiab]

hi! i'm wondering what @gcanti 's thoughts are on how far off the experimental modules are from being no longer experimental. thanks for the great library once again!

[gcanti]

@osdiab my plan would be to make them official in the next major release (which in turn depends on the next major release of fp-ts)

[chrbala]

Is there a reason why D.failure requires a string error message, and can't handle arbitrary types? Does this require the use of the Kleisli module? The current docs aren't clear enough for me to use it vs the Decoder module, which is well documented. I want to include various values including an error code and parameters so that the error string can be localized with the relevant values.

It would also be nice to be able to return multiple errors.

[jamiehodge]

What is the rationale behind encoders not returning an Either? Is there an argument against using E.stringifyJson during encoding?

[jamiehodge]

Another question: there doesn't appear to be a clean way to get the encoded type. ReturnType<typeof Foo.encode> feels crude.

[Frikki]

@gcanti

@williamareynolds I totally forgot to update Schema.md after the last release, thanks for pointing out (should be ok now)

The interpreter refers to MySchemable1 while the rest of the document doesn’t define it.

[chrbala]

Is there a good reason why the input types for decoders tends to be unknown?

e.g.

const s1 = D.string; // D.Decoder<unknown, string>
s1.decode(5); // ok

const s2: D.Decoder<string, string> = D.string;
s2.decode(5); // compiler fails

const o1 = D.type({ s1 }); // D.Decoder<unknown, { s1: string; }>
o1.decode({}); // ok

// Type 'Decoder<string, string>' is not assignable to type 'Decoder<unknown, string>'.
D.type({ s2 });

// D.Decoder<unknown, number>
const t1 = pipe(
  D.string,
  D.parse(value => D.success(Number.parseInt(value, 10)))
);
t1.decode(5); // ok

const t2: D.Decoder<string, number> = pipe(
  D.string,
  D.parse(value => D.success(Number.parseInt(value, 10)))
);
t2.decode(5); // compiler fails

It seems like cases that could be blocked by the compiler should be blocked by the compiler. The input could always be cast to any if the author knows better.

Also, there doesn't seem to be a good way to get the type of the input in a decoder. For the output types, you can do D.TypeOf<typeof t2>, which would yield number in this case, but there's no way to get the string input type. Something like a D.InputTypeOf<typeof t2> would be helpful. Of course, it's only helpful if the input type is refined more than unknown, which is currently quite common, as described above.

[gcanti]

Is there a reason why D.failure requires a string error message, and can't handle arbitrary types? Does this require the use of the Kleisli module?

@chrbala D.failure is just an helper function tailored according to the fixed DecodeError type, if you want to change the error type you may want to use the Kleisli module and define your own Decoder module, see also https://github.com/gcanti/io-ts/issues/478

Is there a good reason why the input types for decoders tends to be unknown?

because when you start decoding something you likely start from an unknown

there doesn't seem to be a good way to get the type of the input in a decoder

you can use D.InputOf

What is the rationale behind encoders not returning an Either?

@jamiehodge prior art I guess, in general encoders don't fail

there doesn't appear to be a clean way to get the encoded type

you can use OutputOf

const NumberToString: E.Encoder<string, number> = {
  encode: String
}

type MyOutputType = E.OutputOf<typeof NumberToString>
/*
type MyOutputType = string
*/

[mmkal]

@gcanti I noticed there's a 3.0.0 branch so wanted to check as I fear I'm a bit behind on the new format. Am I right in understanding that props will be gone completely?

I've worked on a few projects that use props for reflection-like functionality - which can be useful for things like UI generation, and outputting other formats like swagger/openapi/json-schema from io-ts Type instances. Mostly be recursing through props.

If props are going to be removed, do you have a recommended migration path for such tools? Happy to give an example of the kind of "reflection" I'm referring to if it's unclear.

[gcanti]

@mmkal in my intention v3 will contain only the experimental modules.

If props are going to be removed, do you have a recommended migration path for such tools? Happy to give an example of the kind of "reflection" I'm referring to if it's unclear.

Yes please, I can see two possible solution

• extending the built-in Decoder module by adding more meta infos
• using a Schema-like approach

but a concrete example would certainly help to experiment and validate both solutions.

[mmkal]

Sure - this is a minimal version of something like we built on a previous team. We would define server schemas with io-ts, and inspect the io-ts to generate json-schema for use in non-typescript tools (e.g. clients written in other languages, UIs etc.). Not published or anything, as it doesn't cover all cases - it defaults to {}, effectively json-schema's any type. But it worked well:

import * as json from 'json-schema'
import * as t from 'io-ts'

type MappableType =
  | t.NumberType
  | t.StringType
  | t.NullType
  | t.BooleanType
  | t.LiteralType<any>
  | t.KeyofType<any>
  | t.InterfaceType<any>
  | t.PartialType<any>
  | t.UnionType<any>
  | t.ArrayType<any>
  | t.TupleType<any>
  | t.IntersectionType<any>
  | t.RefinementType<any>

export const toJsonSchema = (_type: t.Type<any, any>): json.JSONSchema7 => {
  const type = _type as MappableType
  if (type._tag === 'StringType') {
    return {type: 'string'}
  }
  if (type._tag === 'NumberType') {
    return {type: 'number'}
  }
  if (type._tag === 'NullType') {
    return {type: 'null'}
  }
  if (type._tag === 'BooleanType') {
    return {type: 'boolean'}
  }
  if (type._tag === 'LiteralType') {
    return {const: type.value}
  }
  if (type._tag === 'KeyofType') {
    return {type: 'string', enum: Object.keys(type.keys)}
  }
  if (type._tag === 'UnionType') {
    return {anyOf: type.types.map(toJsonSchema)}
  }
  if (type._tag === 'IntersectionType') {
    return {allOf: type.types.map(toJsonSchema)}
  }
  if (type._tag === 'InterfaceType') {
    return {
      type: 'object',
      required: Object.keys(type.props),
      properties: Object.fromEntries(
        Object.entries<t.Type<any>>(type.props).map(([key, subtype]) => [key, toJsonSchema(subtype)]),
      ),
    }
  }
  if (type._tag === 'PartialType') {
    return {
      type: 'object',
      properties: Object.fromEntries(
        Object.entries<t.Type<any>>(type.props).map(([key, subtype]) => [key, toJsonSchema(subtype)]),
      ),
    }
  }
  if (type._tag === 'ArrayType') {
    return {
      type: 'array',
      items: toJsonSchema(type.type),
    }
  }
  if (type._tag === 'TupleType') {
    return {
      type: 'array',
      items: type.types.map(toJsonSchema),
    }
  }
  if (type._tag === 'RefinementType') {
    if (type.name === 'Int') {
      return {type: 'integer'}
    }
    return {
      ...toJsonSchema(type.type),
      description: 'Predicate: ' + (type.predicate.name || type.name),
    }
  }
  // could add more here for DateFromISOString, etc. etc.
  return unhandledType(type)
}

const unhandledType = (_shouldBeNever: never) => ({})

Example usage:

const type = t.type({
  a: t.string,
  b: t.number,
  c: t.union([t.boolean, t.null]),
  d: t.partial({
    e: t.literal('e'),
    f: t.keyof({f1: 0, f2: 0}),
    i: t.intersection([
      t.type({
        j: t.string,
      }),
      t.partial({
        k: t.Int,
      }),
    ]),
  }),
  l: t.array(t.string),
  m: t.tuple([t.string, t.boolean, t.number]),
})
console.log(JSON.stringify(toJsonSchema(type), null, 2))

output

{
  "type": "object",
  "required": [
    "a",
    "b",
    "c",
    "d",
    "l",
    "m"
  ],
  "properties": {
    "a": {
      "type": "string"
    },
    "b": {
      "type": "number"
    },
    "c": {
      "anyOf": [
        {
          "type": "boolean"
        },
        {
          "type": "null"
        }
      ]
    },
    "d": {
      "type": "object",
      "properties": {
        "e": {
          "const": "e"
        },
        "f": {
          "type": "string",
          "enum": [
            "f1",
            "f2"
          ]
        },
        "i": {
          "allOf": [
            {
              "type": "object",
              "required": [
                "j"
              ],
              "properties": {
                "j": {
                  "type": "string"
                }
              }
            },
            {
              "type": "object",
              "properties": {
                "k": {
                  "type": "integer"
                }
              }
            }
          ]
        }
      }
    },
    "l": {
      "type": "array",
      "items": {
        "type": "string"
      }
    },
    "m": {
      "type": "array",
      "items": [
        {
          "type": "string"
        },
        {
          "type": "boolean"
        },
        {
          "type": "number"
        }
      ]
    }
  }
}

We'd actually use it by defining server routes via objects like:

interface Route {
  method: 'get' | 'post' | 'put' | 'delete'
  path: string
  request: t.Type<...>
  response: t.Type<...>
}

// sample route
const myRoute = {
  method 'put',
  path: '/user',
  request: t.type({
    body: t.type({
      firstName: t.string,
      lastName: t.string,
    }),
  }),
  response: t.type({
    body: t.type({
      id: t.string,
      firstName: t.string,
      lastName: t.string,
    }),
  }),
}      

Then we would inspect those Route objects like above and generate valid openapi that tells clients how they can use the service. IRL we had more cases which handle Int, DateFromISOString, etc. and there were some extra restrictions on what request and response looked like, but hopefully you get the idea. io-ts does the more advanced parsing/validation on the server side, while the openapi is mostly to help clients construct valid requests, so it doesn't matter too much that refinements and types which decode/transform like DateFromISOString weren't fully captured.

Purely from the word "Schema" my guess is the Schema-like approach you mentioned might be useful here, but I'm not sure how exactly the above should be refactored if Type goes away. Would be great to get your thoughts!

[gcanti]

@mmkal the first option (extending the built-in Decoder module by adding more meta infos) would be something along the lines of

import * as assert from 'assert'
import { pipe } from 'fp-ts/function'
import * as R from 'fp-ts/ReadonlyRecord'
import * as D from 'io-ts/Decoder'
import { JSONSchema7 } from 'json-schema'

// -------------------------------------------------------------------------------------
// extensions
// -------------------------------------------------------------------------------------

interface StringDecoder extends D.Decoder<unknown, string> {
  readonly _tag: 'StringDecoder'
}

interface TypeDecoder<A> extends D.Decoder<unknown, A> {
  readonly _tag: 'TypeDecoder'
  readonly props: { [K in keyof A]: D.Decoder<unknown, A[K]> }
}

// etc...

const string: StringDecoder = {
  ...D.string,
  _tag: 'StringDecoder'
}

const type = <A>(properties: { [K in keyof A]: D.Decoder<unknown, A[K]> }): TypeDecoder<{ [K in keyof A]: A[K] }> => ({
  ...D.type(properties),
  _tag: 'TypeDecoder',
  props: properties
})

// etc...

// -------------------------------------------------------------------------------------
// interpreter
// -------------------------------------------------------------------------------------

type MappableType = StringDecoder | TypeDecoder<unknown>

export const toJsonSchema = (_type: D.Decoder<unknown, unknown>): JSONSchema7 => {
  const type = _type as MappableType
  if (type._tag === 'StringDecoder') {
    return { type: 'string' }
  }
  if (type._tag === 'TypeDecoder') {
    return {
      type: 'object',
      required: Object.keys(type.props),
      properties: pipe(type.props, R.map(toJsonSchema))
    }
  }
  // etc...

  // unhandled decoder
  return {}
}

// -------------------------------------------------------------------------------------
// test
// -------------------------------------------------------------------------------------

const decoder = type({
  body: type({
    firstName: string,
    lastName: string
  })
})

assert.deepStrictEqual(toJsonSchema(decoder), {
  type: 'object',
  properties: {
    body: {
      type: 'object',
      properties: {
        firstName: {
          type: 'string'
        },
        lastName: {
          type: 'string'
        }
      },
      required: ['firstName', 'lastName']
    }
  },
  required: ['body']
})

With Schema-like I mean the Schemable / Schema experimental modules.

1. first of all you must implement a Schemable instance for your data type, let's say a JsonSchema<A> data type

// JsonSchema.ts

import * as C from 'fp-ts/Const'
import { pipe } from 'fp-ts/function'
import * as R from 'fp-ts/ReadonlyRecord'
import { JSONSchema7 } from 'json-schema'
import * as S from 'io-ts/Schemable'

// -------------------------------------------------------------------------------------
// model
// -------------------------------------------------------------------------------------

export interface JsonSchema<A> {
  readonly compile: (definitions?: Record<string, JSONSchema7 | undefined>) => C.Const<JSONSchema7, A>
}

// -------------------------------------------------------------------------------------
// constructors
// -------------------------------------------------------------------------------------

export function literal<A extends readonly [S.Literal, ...ReadonlyArray<S.Literal>]>(
  ...values: A
): JsonSchema<A[number]> {
  return {
    compile: () => C.make({ enum: [...values] })
  }
}

// -------------------------------------------------------------------------------------
// primitives
// -------------------------------------------------------------------------------------

export const string: JsonSchema<string> = {
  compile: () => C.make({ type: 'string' })
}

export const number: JsonSchema<number> = {
  compile: () => C.make({ type: 'number' })
}

export const boolean: JsonSchema<boolean> = {
  compile: () => C.make({ type: 'boolean' })
}

// tslint:disable-next-line: readonly-array
export const UnknownArray: JsonSchema<Array<unknown>> = {
  compile: () => C.make({ type: 'array' })
}

export const UnknownRecord: JsonSchema<Record<string, unknown>> = {
  compile: () => C.make({ type: 'object' })
}

// -------------------------------------------------------------------------------------
// combinators
// -------------------------------------------------------------------------------------

const nullJsonSchema: JsonSchema<null> = {
  compile: () => C.make({ enum: [null] })
}

export function nullable<A>(or: JsonSchema<A>): JsonSchema<null | A> {
  return union(nullJsonSchema, or)
}

export function type<A>(properties: { [K in keyof A]: JsonSchema<A[K]> }): JsonSchema<A> {
  return {
    compile: (lazy) =>
      C.make({
        type: 'object',
        properties: pipe(
          properties,
          R.map<JsonSchema<unknown>, JSONSchema7>((p) => p.compile(lazy))
        ),
        required: Object.keys(properties)
      })
  }
}

export function partial<A>(properties: { [K in keyof A]: JsonSchema<A[K]> }): JsonSchema<Partial<A>> {
  return {
    compile: (lazy) =>
      C.make({
        type: 'object',
        properties: pipe(
          properties,
          R.map<JsonSchema<unknown>, JSONSchema7>((p) => p.compile(lazy))
        )
      })
  }
}

export function record<A>(codomain: JsonSchema<A>): JsonSchema<Record<string, A>> {
  return {
    compile: (lazy) =>
      C.make({
        type: 'object',
        additionalProperties: codomain.compile(lazy)
      })
  }
}

// tslint:disable-next-line: readonly-array
export function array<A>(items: JsonSchema<A>): JsonSchema<Array<A>> {
  return {
    compile: (lazy) =>
      C.make({
        type: 'array',
        items: items.compile(lazy)
      })
  }
}

export function tuple<A extends ReadonlyArray<unknown>>(
  ...components: { [K in keyof A]: JsonSchema<A[K]> }
): JsonSchema<A> {
  const len = components.length
  return {
    compile: (lazy) =>
      C.make({
        type: 'array',
        items: len > 0 ? components.map((c) => c.compile(lazy)) : undefined,
        minItems: len,
        maxItems: len
      })
  }
}

export const intersect = <B>(right: JsonSchema<B>) => <A>(left: JsonSchema<A>): JsonSchema<A & B> => ({
  compile: (lazy) => C.make({ allOf: [left.compile(lazy), right.compile(lazy)] })
})

export function sum<T extends string>(
  _tag: T
): <A>(members: { [K in keyof A]: JsonSchema<A[K] & Record<T, K>> }) => JsonSchema<A[keyof A]> {
  return (members: Record<string, JsonSchema<unknown>>) => {
    return {
      compile: (lazy) => C.make({ anyOf: Object.keys(members).map((k) => members[k].compile(lazy)) })
    }
  }
}

export function lazy<A>(id: string, f: () => JsonSchema<A>): JsonSchema<A> {
  const $ref = `#/definitions/${id}`
  return {
    compile: (definitions) => {
      if (definitions !== undefined) {
        if (definitions.hasOwnProperty(id)) {
          return C.make({ $ref })
        }
        definitions[id] = undefined
        return (definitions[id] = f().compile(definitions))
      } else {
        definitions = { [id]: undefined }
        definitions[id] = f().compile(definitions)
        return C.make({
          definitions,
          $ref
        })
      }
    }
  }
}

export function union<A extends readonly [unknown, ...ReadonlyArray<unknown>]>(
  ...members: { [K in keyof A]: JsonSchema<A[K]> }
): JsonSchema<A[number]> {
  return {
    compile: (lazy) => C.make({ anyOf: members.map((m) => m.compile(lazy)) })
  }
}

// -------------------------------------------------------------------------------------
// instances
// -------------------------------------------------------------------------------------

export const URI = 'io-ts/JsonSchema'

export type URI = typeof URI

declare module 'fp-ts/lib/HKT' {
  interface URItoKind<A> {
    readonly [URI]: JsonSchema<A>
  }
}

export const Schemable: S.Schemable1<URI> & S.WithUnknownContainers1<URI> & S.WithUnion1<URI> = {
  URI,
  literal,
  string,
  number,
  boolean,
  UnknownArray,
  UnknownRecord,
  nullable,
  type,
  partial,
  record,
  array,
  tuple: tuple as S.Schemable1<URI>['tuple'],
  intersect,
  sum,
  lazy,
  union: union as S.WithUnion1<URI>['union']
}

now you can define a schema...

import * as J from './JsonSchema'
import { make, interpreter } from 'io-ts/Schema'

const schema = make((S) =>
  S.type({
    body: S.type({
      firstName: S.string,
      lastName: S.string
    })
  })
)

...which can be converted into a JsonSchema...

const jsonSchema = interpreter(J.Schemable)(schema)

import * as assert from 'assert'

assert.deepStrictEqual(jsonSchema.compile(), {
  type: 'object',
  properties: {
    body: {
      type: 'object',
      properties: {
        firstName: {
          type: 'string'
        },
        lastName: {
          type: 'string'
        }
      },
      required: ['firstName', 'lastName']
    }
  },
  required: ['body']
})

... or a Decoder...

import * as D from 'io-ts/Decoder'
import * as E from 'fp-ts/Either'
import { pipe } from 'fp-ts/function'

const decoder = interpreter(D.Schemable)(schema)

console.log(
  pipe(
    decoder.decode({}),
    E.bimap(D.draw, () => 'ok!')
  )
)
/*
{
  _tag: 'Left',
  left: 'required property "body"\n' +
    '└─ cannot decode undefined, should be Record<string, unknown>'
}
*/

or an Eq, a Guard, a Codec, a TaskDecoder, an Arbitrary, etc...

What I also like about the second option is that is statically checked (i.e. the type checker raises an error if the schema cannot be fully interpreted), while with the first option you are forced to return {}

[mmkal]

@gcanti thank you, that is extremely helpful. I agree that the second approach looks better. A couple of follow up questions:

• is the make(S => ...) pattern a necessity? For large existing codebases that are currently using plain t.type(...) it could be a painful upgrade. And it's a little more confusing on first look. The t.type(...), t.string usages were very intuitive to read, even for newcomers. Is there a way to get a reference to that S variable outside of the make scope, maybe? Or build schemas that can later be plugged into it. That way maybe the details can be hidden by developers who just want a helper for writing a schema. Not sure if this question makes sense, I haven't quite got my head around the generics (not quite clear to me what the A in Schemable<A> represents).
• with the Schema-like approach, how should custom types like DateFromISOString be handled? In the old world, we checked type.name === 'DateFromISOString' and rendered a date picker for those fields (for a similar tool which generates a UI from io-ts schemas). Similarly, refinement types?
• I'm not sure I follow why fp-ts/Const is used - would it be possible to build a schema for JsonSchema7 rather than Const<JsonSchema7>?

[mmkal]

Oh, one other pattern I'm unclear on how to migrate is to take somebody else's type and create a new one from it:

const User = t.type({
  name: t.string,
  address: t.string
})

// Somewhere else:

const UserWithAge = t.type({
  ...User.props,
  age: t.number
})

[mlegenhausen]

@mmkal even in the first version this was normally solved via t.intersection which is seems to be available in v2 too?

const UserWithAge = t.intersection([User, t.type({ age: t.number })])

[mmkal]

True, maybe I could have given a better example like:

const {name, ...rest} = User.props

const NamelessUser = t.type(rest)

[gcanti]

is the make(S => ...) pattern a necessity?

@mmkal yes, you need the Schemable interface (aka type class) in scope in order to

• use its operations (e.g. string, type, tuple, etc...)
• choose a concrete instance later (e.g. Schemable from Decoder.ts, Schemable from Guard.ts, etc...)

how should custom types like DateFromISOString be handled?

You can extend Schemable with your own custom schemable interface, see https://github.com/gcanti/io-ts/blob/f13a10ae9d405f3fb8564cf3b7917d31aa7c0e27/Schema.md#how-to-extend-the-built-in-schema

I'm not sure I follow why fp-ts/Const is used

That's because I need a type constructor of kind * -> * (at least), like Guard<A> or Eq<A>, in order to define a Schemable instance.

const {name, ...rest} = User.props

const NamelessUser = t.type(rest)

AFAIK that's not possible

[MoSheikh]

Firstly, thank you for such a great project. My only concern with this direction is this layer of customization required for extending anything default.

For example, in my use case I am decoding objects into Date and MongoDB ObectId classes coming in from REST requests. Previously, I could define a name, guard, decoder, and toString() method for each of the classes to create the type that would fit my use cases.

Now, to do the same thing, I would have to do the following according to the Schema module readme:

1. Declare MySchemable1, add HKT property for each desired class
2. Declare MySchemable2, add Kind property for each desired class
3. Declare MySchemable3, add Kind2 property for each desired class
4. Declare MySchema
5. Make constructor for MySchema
6. Create instance of MySchemable2 with
7. Create Interpreter
8. Use interpreter to create desired Guard / Decoder / etc...

It seems to me that this direction for the API essentially adds a lot more boilerplate that must also be extended and maintained as you add additonal Schemable types. I'm not the smartest person so maybe I'm missing something about why the schemas are being migrated to in the first place.

If anyone would like to provide some advice / insight into these concerns, I would immensely appreciate it. Thank you.

[gcanti]

@MoSheikh the Schemable / Schema modules are useful if you are interested in defining generic schemas a then compile them down to different targets, but you don't have to: assuming you are only interested in decoding you can simply use the Decoder module and define your own decoders.

For example:

in my use case I am decoding objects into Date

import { pipe } from 'fp-ts/function'
import * as D from 'io-ts/Decoder'

export const MyDateDecoder: D.Decoder<unknown, Date> = pipe(
  D.string,
  D.compose({
    decode: (s) => {
      const d = new Date(s)
      return isNaN(d.getTime()) ? D.failure(s, 'not a Date') : D.success(d)
    }
  })
)

Another example: assuming you are only interested in both decoding and encoding you can just use the Codec module and define your own codecs

import * as C from 'io-ts/Codec'

export const MyDateCodec: C.Codec<unknown, string, Date> = pipe(
  C.string,
  C.compose({
    decode: (s) => {
      const d = new Date(s)
      return isNaN(d.getTime()) ? D.failure(s, 'not a Date') : D.success(d)
    },
    encode: (d) => d.toISOString()
  })
)

[gcanti]

What do you think about https://github.com/gcanti/io-ts/issues/525?