Fp-ts: [RFC] Combining `Either`s with different error types

[gcanti]

The problem

// example from https://github.com/natefaubion/purescript-checked-exceptions

import * as E from 'fp-ts/lib/Either'
import { pipe } from 'fp-ts/lib/pipeable'

type HttpError = 'HttpError'
declare function get(url: string): E.Either<HttpError, string>
type FsError = 'FsError'
declare function write(path: string, content: string): E.Either<FsError, void>

type Err = HttpError | FsError

const program = pipe(
  get('http://purescript.org'), // error: Type '"HttpError"' is not assignable to type '"FsError"'
  E.chain(content => write('~/purescript.html', content))
)

This is because E.chain is inferred as

//                                                                   relevant type here ---v
chain<"FsError", string, void>(f: (a: string) => E.Either<"FsError", void>): (ma: E.Either<"FsError", string>) => E.Either<"FsError", void>

Solutions?

1) explicit cast

const writeE1: (path: string, content: string) => E.Either<Err, void> = write

const program1 = pipe(
  get('http://purescript.org'),
  E.chain(content => writeE1('~/purescript.html', content))
)

2) explicit mapLeft

const widen = E.mapLeft<Err, Err>(e => e)

const program2 = pipe(
  get('http://purescript.org'),
  E.chain(content =>
    pipe(
      write('~/purescript.html', content),
      widen
    )
  )
)

3) lifting with flow and mapLeft

import { flow } from 'fp-ts/lib/function'

const writeE2 = flow(
  write,
  widen
)

const program3 = pipe(
  get('http://purescript.org'),
  E.chain(content => writeE2('~/purescript.html', content))
)

4) defining a chain overload that combines the error types

const flatMap: <E2, A, B>(
  f: (a: A) => E.Either<E2, B>
) => <E1>(ma: E.Either<E1, A>) => E.Either<E1 | E2, B> = E.chain as any

const program4 = pipe(
  get('http://purescript.org'),
  flatMap(content => write('~/purescript.html', content))
)

Others?

[sledorze]

@gcanti if overloads were play better with currying, we maybe would have been able to add some constraint on the pipe Type and achieve something like a pipe2C1..
Maybe there's some clever way to achieve this..

problem with currying and overloads is that the first matching overload is taken (the one necessary for the constraint to be expressed) this prevents actually doing any overloads matching..

[sledorze]

App developper perspective here:

Sometimes we want the error to widen but would like to control it at some boundaries.
Ability to understand from where an unwanted widening has happened is crucial.

From earlier testing, flatMap is the best one that fits that bill here.
The widening conflict can be tracked using hover on the pipe stages, it's pretty readable.

[raveclassic]

Sometimes you want coproduct on the left (RemoteData, Either etc.) and sometimes product (Reader etc.). It depends on the position of the left value - covariant or contravariant.

I've ended up with a naive (and pretty ugly) solution. If you can "build" coproduct/product left for two values of the type, then you can actually combine more than two.

Hope that helps, we find such "combine"s very useful in our projects.

UPD: If such solution fits fp-ts design I could send a PR.

[sledorze]

@raveclassic you're right, btw explicit variance has been a topic in latest design meeting..
https://github.com/microsoft/TypeScript/issues/32157

[leighman]

4 is very nice to use, feels very TypeScript-y and is what I'd expect if I was coming to fp-ts not having used Haskell/PureScript

[mattiamanzati]

Option 5.
Define a utility function that works at type level for every instance of Chain, so that there is no additional boilerplate to define a Chain instance.

import * as E from 'fp-ts/lib/Either'
import { pipe } from 'fp-ts/lib/pipeable'
import { URIS2, Kind2, Kind } from 'fp-ts/lib/HKT';
import { Chain2C, Chain2 } from 'fp-ts/lib/Chain';

type HttpError = 'HttpError'
declare function get(url: string): E.Either<HttpError, string>
type FsError = 'FsError'
declare function write(path: string, content: string): E.Either<FsError, void>

type Err = HttpError | FsError


function flatMap<U extends URIS2>(M: Chain2<U>): <E2, A, B>(
    f: (a: A) => Kind2<U, E2, B>
  ) => <E1>(ma: Kind2<U, E1, A>) => Kind2<U, E1 | E2, B> {
      return M.chain as any
  }


const program5 = pipe(
    get('http://purescript.org'),
    flatMap(E.either)(content => write('~/purescript.html', content))
  )

[raveclassic]

@mattiamanzati This won't work for Readers because they require a & on the left.

function flatMap<U extends URIS2>(
    M: Chain2<U>,
): <E2, A, B>(f: (a: A) => Kind2<U, E2, B>) => <E1>(ma: Kind2<U, E1, A>) => Kind2<U, E1 | E2, B> {
    return f => ma => M.chain(ma, f as any) as any;
}

const flatMapReader = flatMap(reader);
const a = asks((e: { a: string }) => e.a.toUpperCase());
const b = asks((e: { b: number }) => e.b.toString());
const r = flatMapReader(() => b)(a);
r({ a: '123' }); // TypeError: Cannot read property 'toString' of undefined

[mattiamanzati]

@raveclassic indeed! But I think that the decision should be made by the developer, there is no "default behaviour" that wont be sensible here.
We can have 2 different methods to achive that, or use a type-level dictionary to choose between them.

import * as E from 'fp-ts/lib/Either'
import * as R from 'fp-ts/lib/Reader'
import { pipe } from 'fp-ts/lib/pipeable'
import { URIS2, Kind2, Kind } from 'fp-ts/lib/HKT';
import { Chain2C, Chain2 } from 'fp-ts/lib/Chain';

type HttpError = 'HttpError'
declare function get(url: string): E.Either<HttpError, string>
type FsError = 'FsError'
declare function write(path: string, content: string): E.Either<FsError, void>

type Err = HttpError | FsError

// TODO: better naming! this could be flatMap
function flatMapOr<U extends URIS2>(
    M: Chain2<U>,
): <E2, A, B>(f: (a: A) => Kind2<U, E2, B>) => <E1>(ma: Kind2<U, E1, A>) => Kind2<U, E1 | E2, B> {
    return f => ma => M.chain(ma, f as any) as any;
}
// and this something other!
function flatMapAnd<U extends URIS2>(
    M: Chain2<U>,
): <E2, A, B>(f: (a: A) => Kind2<U, E2, B>) => <E1>(ma: Kind2<U, E1, A>) => Kind2<U, E1 & E2, B> {
    return f => ma => M.chain(ma, f as any) as any;
}
// ... or we can do something like flatMap(R.reader) and have an interface (like HKT) 
// that returns either "&" or "|" to decide the correct operation. Not recommended though.

const flatMapReader = flatMapAnd(R.reader);
const a = R.asks((e: { a: string }) => e.a.toUpperCase());
const b = R.asks((e: { b: number }) => e.b.toString());
const r = flatMapReader(() => b)(a);
r({ a: '123', b: 123 }); // TypeError: Cannot read property 'toString' of undefined

const program5 = pipe(
    get('http://purescript.org'),
    flatMapOr(E.either)(content => write('~/purescript.html', content))
  )

[mattiamanzati]

Maybe that could be decided by knowing if the "L" type appears in variant or contravariant position, but still, not sure if that's a great default.

[raveclassic]

That's better but the typechecker still cannot help the developer to choose the right helper: flatMapOr or flatMapAnd. Take a look - this is the only way I found to set some constraint on how left sides should be combined for two containers. I think it's possible to do the same for Chain.

[mattiamanzati]

Yeah, you're basically pre-defining defaults for each Chain instance by using either product or coproduct left in the type instance, right?

[raveclassic]

Yep, smth like this:

declare function flatMapOr<M extends URIS2>(M: Chain2<M> & CoproductLeft<M>): <EA, A, EB, B>(ma: Kind2<M, EA, A>, f: (a: A) => Kind2<M, EB, B>) => Kind2<M, EA | EB, B>;
declare function flatMapAnd<M extends URIS2>(M: Chain2<M> & ProductLeft<M>): <EA, A, EB, B>(ma: Kind2<M, EA, A>, f: (a: A) => Kind2<M, EB, B>) => Kind2<M, EA & EB, B>;

[mattiamanzati]

And are we sure that there are'nt instances of Chain where both & and | are legal operations?

[raveclassic]

Can't think of any :)

[christianbradley]

This, IMO, is a very important thing to figure out. I've been using fp-ts consistently for years in major products, and one of the most valuable tools is chaining methods that return Either/TaskEither. I've gotten into the (anti?)pattern of starting my chains with the initial param "Right" value but explicit sum type for all possible "Left" values in the chain... ie:

taskEither.fromIO<InvalidEnv | InitFailed | FetchAccessTokenFailed | ..., NodeJS.ProcessEnv>(
  new IO(() => process.env)
)
  .chain(validateEnv)
  .chain(initializeApp)
  .chain(fetchAccessToken)
  ...

with 2.x out that doesn't even work any more, as chaining through pipe fails when using sum types over methods that return union members (see #1028 )

I'm all for a method that widens the "Left" types on either - perhaps separate from chain?
maybe chainAndWiden?

[mlegenhausen]

Maybe I am a little bit uncool from a FP perspective :wink: but I use a wrapper union type for all possible errors I want to combine. Instead of widen you can then use a simple mapLeft.

import { Union, of } from 'ts-union'

const AppError = Union({
  FsError: of<FsError>(),
  HttpError: of<HttpError>()
})
type AppError = typeof AppError.T

const program: Either<AppError, void> = pipe(
  get('http://purescript.org'),
  E.mapLeft(AppError.HttpError),
  E.chain(content => pipe(
    write('~/purescript.html', content),
    E.mapLeft(AppError.FsError)
  ))
)

What I find beneficial is that the error represents the path where the error happend because in bigger application I wrap error wrapper types in other error wrapper types. Also are these type extendable. So you can add every metadata you want.

[christianbradley]

Another thought here...

I think the explicit mapLeft solution should be avoided, as this can lead to a number of unnecessary method calls that do nothing more than an identity function in the compiled script.

The flatMapOr / flatMapAnd solution looks pretty darn close to me. Definitely want something that works for all Chainable Types. Naming is the only thing I'm unsure about. Are there any analogs in the Haskell/Scala/Purescript world we can work with?

[steida]

This very confusing behavior needs to be documented somehow. I bet it's a show stopper and deal-breaker for many fp-ts users. This is the situation where types blocks doing anything meaningful.

As I see it. The possible solutions are

1) Having uber union kitchen sink error type (https://github.com/gcanti/fp-ts/issues/904#issuecomment-558528906) for a bunch of unrelated functions.
2) Typing everything everywhere and hoping TS will magically work (I know there is a "simple" rule how TS evaluates things, but who knows it?) After few chains, types are everywhere.
3) There are no other options except disabling type checking at all.

[raveclassic]

@steida

There are no other options except disabling type checking at all.

Actually we have a working solution described in this comment and starting with this one. I'm not sure if it should be added to the core (it's up to @gcanti to decide) but it's battletested in production in several our projects.

[gcanti]

The official way to do it is what people do in other languages (Haskell, PureScript, Scala) and what @mlegenhausen is doing in TypeScript: wrap (and / or transform) the sub-errors into a new sum type.

So given the following modules

// module1.ts
import * as E from 'fp-ts/lib/Either'
export type Err = { type: 'a' } | { type: 'b' }
export declare function api1(x: string): E.Either<Err, number>
export declare function api2(x: string): E.Either<Err, void>

// module2.ts
import * as E from 'fp-ts/lib/Either'
export type Err = { type: 'a' } | { type: 'c' }
export declare function api3(x: number): E.Either<Err, boolean>
export declare function api4(x: boolean): E.Either<Err, string>

the pattern is

// index.ts

import * as E from 'fp-ts/lib/Either'
import { flow } from 'fp-ts/lib/function'
import { pipe } from 'fp-ts/lib/pipeable'
import * as M1 from './module1'
import * as M2 from './module2'

// sum type
type Err = { type: 'M1'; err: M1.Err } | { type: 'M2'; err: M2.Err }

// constructors
const m1 = (err: M1.Err): Err => ({ type: 'M1', err })
const m2 = (err: M2.Err): Err => ({ type: 'M2', err })

// lifting functions
const api1 = flow(M1.api1, E.mapLeft(m1))
const api2 = flow(M1.api2, E.mapLeft(m1))
const api3 = flow(M2.api3, E.mapLeft(m2))
const api4 = flow(M2.api4, E.mapLeft(m2))

// new API
export function api5(x: string): E.Either<Err, void> {
  return pipe(api1(x), E.chain(api3), E.chain(api4), E.chain(api2))
}

[raveclassic]

@gcanti Yeah, on the part of lib API and its consistency that's perfect, although ugly on DX part.

Thinking further about API (flatMap & co), I think I can't find any truly generic solution because structures like ReaderTaskEither<R, E, A> require both & for R type arg and | for E. So that specifically for RTE the signature of flatMap should rather be:

flatMap: <RA, EA, A, RB, EB, B>(fa: ReaderTaskEither<R, E, A>, f: (a: A) => ReaderTaskEither<RB, EB, B>) => ReaderTaskEither<RA & RB, EA | EB, B>

[raveclassic]

Another thought - interesting what could we do if we had explicit variance in TS 😄 :

interface FlatMap1<F extends URIS> extends Chain1<F> {
    //Option
    flatMap<+A, +B>(fa: Kind<F, A>, f: (a: A) => Kind<F, B>): Kind<F, A | B>
}
interface FlatMap2<F extends URIS2> extends Chain2<F> {
    //Reader
    flatMap<-EA, +A, -EB, +B>(fa: Kind2<F, EA, A>, f: (a: A) => Kind2<F, EB, B>): Kind2<F, EA & EB, A | B>
    //Either, RemoteData
    flatMap<+EA, +A, +EB, +B>(fa: Kind2<F, EA, A>, f: (a: A) => Kind2<F, EB, B>): Kind2<F, EA | EB, A | B>
}
interface FlatMap3<F extends URIS3> extends Chain3<F> {
    //ReaderTaskEither
    flatMap<-RA, +EA, +A, -RB, +EB, +B>(
        fa: Kind3<F, RA, EA, A>,
        f: (a: A) => Kind3<F, RB, EB, B>,
    ): Kind3<F, RA & RB, EA | EB, A | B>
}

[sledorze]

@raveclassic you may find a RWC using that pattern in this effect lib:

https://github.com/mikearnaldi/matechs-effect/blob/master/packages/effect/src/overload.ts#L152

[mlegenhausen]

@gcanti thanks for clarification. Seems to be the same pattern that I described only that I use a utility library for creating the new Err sum type.

[minedeljkovic]

The official way to do it is what people do in other languages (Haskell, PureScript, Scala)...

Speaking of Scala (more specifically Scala 3 which adds union support similar to TS), it seems that people there are leaning towards solution 4.. Example link:
https://fp-tower.github.io/2020-01-27-introducing-error-reporting-in-optics/

Did anyone find any specific downside of solution 4.?

Furthermore, if solution 4. is acceptable, I think it could be applied to success channel also. For example, I usually struggle with code like this when integrating fp-ts code with some api that does not use Option for nullability:

import * as E from 'fp-ts/lib/Either'

declare var fa: Either<string, number>
const a: number | null = E.getOrElse(_ => null)(fa) // TS error: ...Type 'number' is not assignable to type 'null'...

With solution 4. this is ok:

import * as E from 'fp-ts/lib/Either'

declare function getOrElse1<E, A1>(onLeft: (e: E) => A1): <A>(ma: Either<E, A>) => A | A1

declare var fa: Either<string, number>
const a: number | null = getOrElse1(_ => null)(fa) // OK

[minedeljkovic]

Solution 4. seems to go nicely with monomorphic types, but it can get tricky with polymorphic types. Same example but with Option on the left:

import * as E from 'fp-ts/lib/Either'
import * as O from 'fp-ts/lib/Option'
import { pipe } from 'fp-ts/lib/pipeable'

type HttpError = 'HttpError'
declare function get(url: string): E.Either<O.Option<HttpError>, string>
type FsError = 'FsError'
declare function write(path: string, content: string): E.Either<O.Option<FsError>, void>

const flatMap: <E2, A, B>(
  f: (a: A) => E.Either<E2, B>
) => <E1>(ma: E.Either<E1, A>) => E.Either<E1 | E2, B> = E.chain as any

const program4 = pipe(
  get('http://purescript.org'),
  flatMap(content => write('~/purescript.html', content)),
)

In this example O.None | O.Some<"HttpError"> | O.Some<"FsError"> type is infered on the left, and AFAIU that is a tricky type to work with:

const program4Ex = pipe(
  program4,
  E.mapLeft(optErr => pipe(
    optErr, // error: Type '"HttpError"' is not assignable to type '"FsError"'
    O.map(err => { /* convert err to something */ }),
  )),
)

[Lonli-Lokli]

How this union errors now are used in the end? Eg we have union of 6 possible errors, how program supposed to work with them when any of them happens?

[kirit0s]

What about Do notation from fp-ts-contrib? I used flatMap from solutoin 4 in pipe. But how better to use this in Do notaion? I don't quite understand how I can write without custom errors (extends Error).

[gcanti]

Thank you all for your comments, looks like we all agree that an additional, more flexible version of chain (i.e. solution 4) would be idiomatic e pretty useful, so here's a proposal

Either

export declare const chainW: <D, A, B>(f: (a: A) => Either<D, B>) => <E>(ma: Either<E, A>) => Either<E | D, B>

export declare const getOrElseW: <E, B>(onLeft: (e: E) => B) => <A>(ma: Either<E, A>) => A | B

Reader

export declare const chainW: <Q, A, B>(f: (a: A) => Reader<Q, B>) => <R>(ma: Reader<R, A>) => Reader<R & Q, B>

ReaderEither

export declare const chainW: <Q, D, A, B>(f: (a: A) => ReaderEither<Q, D, B>) => <R, E>(ma: ReaderEither<R, E, A>) => ReaderEither<R & Q, E | D, B>

export declare const getOrElseW: <Q, E, B>(onLeft: (e: E) => Reader<Q, B>): <R>(ma: ReaderEither<R, E, A>) => Reader<R & Q, A | B>

etc...

[raveclassic]

@gcanti Looks good but what about sequenceTW/sequenceSW/sequenceW/traverseW etc? They seem to be as useful as chain since it's a very common task to combine multiple things in one. This may quickly get out of control.

[christianbradley]

@raveclassic -

1. I think there's a big difference here between sequence/traverse and chain/chainW. Chain is a "flow" based operation that is commonly used nearly everywhere in functional code to replace try/throw/catch (or Promise.reject in the case of TaskEither). The move to a more purely functional methodology by removing classes for types in v2.x has made this even more tedious.

2. Doesn't seem like anyone's banging the door down for sequenceW at the moment. If it becomes a highly requested item, what's wrong with implementing it?

In the end, a library like this needs to find the middle ground between perfectionism and usefulness. I'm all about it. Let's get it in there.

@gcanti - Yes, Please! BTW - what is the meaning of the W suffix? "Widen"?

[patroza]

@christianbradley @raveclassic I was thinking the same thing but couldn't put it so well into words.

@ganti - yes please indeed!

[raveclassic]

@christianbradley

1. Well, there's not that much difference between these two combinators. It's like to say that rxjs switchMap (chain) is used more often than combineLatest (sequenceT).
2. I am :) nothing is wrong, but nothing is also ok in inconsistent API, where something is in the lib and something is not while things should be consistent

[gcanti]

@raveclassic @giogonzo the problem with sequenceTW / sequenceSW is how to type them.

what is the meaning of the W suffix? "Widen"?

@christianbradley yes

[raveclassic]

@gcanti Either each module should have it's own specific implementation:

import { NonEmptyArray } from 'fp-ts/lib/NonEmptyArray'
import { Either } from 'fp-ts/lib/Either'
import { Reader } from 'fp-ts/lib/Reader'

type Intersection<A> = (A extends any ? (a: A) => void : never) extends (a: infer A) => void ? A : never

interface SequenceTWEither {
  <T extends NonEmptyArray<Either<any, any>>>(...args: T): Either<
    { [I in keyof T]: [T[I]] extends [Either<infer E, any>] ? E : never }[number],
    { [I in keyof T]: [T[I]] extends [Either<any, infer A>] ? A : never }
  >
}

declare const sequenceTWEither: SequenceTWEither
declare const a: Either<'Error1', number>
declare const b: Either<'Error2', string>
const r1 = sequenceTWEither(a, b) // Either<'Error1' | 'Error2', [number, string]>

interface SequenceTWReader {
  <T extends NonEmptyArray<Reader<any, any>>>(...args: T): Reader<
    Intersection<
      {
        [I in keyof T]: [T[I]] extends [Reader<infer E, any>] ? E : never
      }[number]
    >,
    { [I in keyof T]: [T[I]] extends [Reader<any, infer A>] ? A : never }
  >
}

declare const sequenceTWReader: SequenceTWReader
declare const c: Reader<{ env1: string }, number>
declare const d: Reader<{ env2: number }, string>
const r2 = sequenceTWReader(c, d) // Reader<{ env1: string } & { env2: number }, [number, string]>

or we could place generic types to Apply module:

import { NonEmptyArray } from 'fp-ts/lib/NonEmptyArray'
import { Kind2, URIS2 } from 'fp-ts/lib/HKT'
import { URI as URIReader } from 'fp-ts/lib/Reader'
import { URI as URIEither } from 'fp-ts/lib/Either'

type Intersection<A> = (A extends any ? (a: A) => void : never) extends (a: infer A) => void ? A : never
interface SequenceTOr<F extends URIS2> {
  <T extends NonEmptyArray<Kind2<F, any, any>>>(...args: T): Kind2<
    F,
    { [I in keyof T]: [T[I]] extends [Kind2<F, infer E, any>] ? E : never }[number],
    { [I in keyof T]: [T[I]] extends [Kind2<F, any, infer A>] ? A : never }
  >
}
interface SequenceTAnd<F extends URIS2> {
  <T extends NonEmptyArray<Kind2<F, any, any>>>(...args: T): Kind2<
    F,
    Intersection<
      {
        [I in keyof T]: [T[I]] extends [Kind2<F, infer E, any>] ? E : never
      }[number]
    >,
    { [I in keyof T]: [T[I]] extends [Kind2<F, any, infer A>] ? A : never }
  >
}

declare const sequenceTWEither: SequenceTOr<URIEither>
declare const sequenceTWReader: SequenceTAnd<URIReader>

[gcanti]

I'm not sure we can trust this "hack"

type UnionToIntersection<U> = (U extends any ? (k: U) => void : never) extends ((k: infer I) => void) ? I : never;

Related: https://github.com/microsoft/TypeScript/issues/29594

[mikearnaldi]

If UnionToIntersection breaks I am probably going to hire a hitman, for the purpose of this discussion an instance of either that mangle errors can be found here:
https://github.com/Matechs-Garage/matechs-effect/blob/master/packages/prelude/src/either.ts

I have more overloading specific to effect but the base shows what's needed to be extended specifically to either

[gcanti]

PR: https://github.com/gcanti/fp-ts/pull/1198

Something missing?

[giogonzo]

A bit late on this, but in our projects I'm also using chainEitherKW (PRed here), e.g. to parse the response body of an api call using io-ts and just ignore the error / keep unknown:

declare function fetchAPI(...): TaskEither<unknown, unknown>

pipe(
  fetchAPI(...),
  chainEitherKW(ResponseType.decode)
)

[christianbradley]

@gcanti and everyone else who contributed, just wanted to say, great work bringing this in. It's made my life so much easier!!!