Ts-toolbelt: Union.StrictOptional

[thesunny]

🍩 Feature Request

Is your feature request related to a problem?

Yes. I am using Union.Strict to create a safe version of a table insert from a schema where an account must discriminate on a type literal. For example (not real), if the type column was user, it would allow the email column to be populated. If the type column was team, it would allow the memberCount column to be populated.

Uses Union.Strict gives the right answer:

type Schema =
  | { name: string, type: "user", email: string }
  | { name: string, type: "team", memberCount: number }
type InsertType =
  | { type: "user", email: string, memberCount?: undefined }
  | { type: "team", email?: undefined, memberCount: number }

There is no similar generic to handle an update. In this case, the update would require the discriminant to be provided in order to complete the discriminated field.

type Schema =
  | { name: string, type: "user", email: string }
  | { name: string, type: "team", memberCount: number }
type UpdateType =
  | { name?: string, type: "user", email?: string, memberCount?: undefined }
  | { name?: string, type: "team", email?: undefined, memberCount?: number }
  | { name?: string, type?: undefined, email?: undefined, memberCount?: undefined }

The discriminant can be determined by identifying the literals (which can conveniently be found in the newly added Any.Literal method!)

Describe the solution you'd like

Would be grateful for there to be a Union.PartialStrict method which did this magically.

Describe alternatives you've considered

I tried writing this but I'm getting stuck on some of the TypeScript around unions and updating each member of the union independently (they seem to merge so that I end up with type: "user" | "email" instead of keeping them separate). I don't have a solution yet.

Teachability, Documentation, Adoption, Migration Strategy

StrictPartial is like Strict but it allows partial completion while still allowing the discriminants to properly discriminate properties on the object. In order for this to work, to update any discriminated properties, the discriminant must be provided.

[millsp]

1: So if I understood well, you determine the fields to make undefined based on the schema that is be queried?

2: Why not remove the fields that are useless, is it to keep it generic (which is great)?
memberCount?: undefined is not needed if it's a type: "user"

3: In your second example, what is the meaning of, why have this?
{ name?: string, type?: undefined, email?: undefined, memberCount?: undefined }

instead of just:

  | { name?: string, type: "user", email?: string, memberCount?: undefined }
  | { name?: string, type: "team", email?: undefined, memberCount?: number }

4: If you ended up with type: "user" | "email" its probably because you forgot to distribute over the parent union:

type t0 = UpdateType extends unknown
          ? // do your stuff here
          : never

This will force a union to get distributed.

[thesunny]

You are correct about 2! It's over-specified. The original reason was that it allowed the setting of null values in some place (i.e. it was something like email?: undefined | null) but as I was playing around with code, I removed the null for simplicity in thinking (fix a smaller problem first); however, I think it needs to be back in there with some more thinking or we can get inconsistent data.

For example, if we had a record:

const record = {name: "john", type: "user", email: "[email protected]"}

// And we updated with

const update = {type: "team", memberCount: 10}

// We'd end up with

const updatedRecord = {name: "john", type: "team", email: "[email protected]", memberCount: 10}

I think I have to spec this a little better...

With respect to 3, within the limits of the conversation without any null, it probably could be reduced to:

  | { name?: string, type: "user", email?: string }
  | { name?: string, type: "team", memberCount?: number }
  | { name?: string }

The third line was to allow name to be set without type since it's not being discriminated against.

But also, thank you so much for that tip about distributing over unions. There is so much undocumented (or poorly documented) stuff in TypeScript and precious few experts at it.

EDIT: Have to be honest, I'm not sure if the answer will belong in a general toolset. Maybe if you give me a day or so, I'll be able to provide a bit more thought around this. One thing I notice though is that the typing around database/validation is pretty poor in general and I wonder if better toolsets specialized for db might help. The only one db types that works pretty well IMO prisma. I was also expecting some types on ajv for json-schema but it's basically untyped except to suppress errors. Typing on superstruct wasn't there which I added custom types which I posted in their issues to get at least some types into it.

[thesunny]

This is why you never explain things when you are tired. Let's retcon that last post.

The reason why I needed the undefined was because I was allowing extra arbitrary properties to pass through.

This...

type UpdateType =
  | { name?: string, type: "user", email?: string, memberCount?: undefined }
  | { name?: string, type: "team", email?: undefined, memberCount?: number }
  | { name?: string, type?: undefined, email?: undefined, memberCount?: undefined }

Would ultimately be joined with { [key:string]: any }

type UpdateType = (
  | { name?: string, type: "user", email?: string, memberCount?: undefined }
  | { name?: string, type: "team", email?: undefined, memberCount?: number }
  | { name?: string, type?: undefined, email?: undefined, memberCount?: undefined }
) & { [key:string]: any }

The ?: undefined were there so that the { [key:string]: any} wouldn't allow in a value for the discriminated properties. This is like Union.Strict which adds the undefined properties but this does the same thing for partials. The third row is to not allow entries into the discriminated properties when there is no discriminant present.

[thesunny]

I built a version of the function as a reference. The idea of StrictPartial is generic but it is specific in that it handles discriminants in a specific way as to prevent bad states.

type Schema =
  | { type: "user"; user_id: number }
  | { type: "email"; email: string; auth_token: string }

// prettier-ignore
type Flatten<T> =
  T extends infer U ? { [K in keyof U]: U[K] } : never

type MapUndefinedToOptional<T extends object> = T extends unknown
  ? Object.Optional<T, Object.SelectKeys<T, undefined>>
  : never

// prettier-ignore
type MapNonLiteralToIncludeUndefined<T extends object> = {
  [K in keyof T]:
    Any.IsLiteral<T[K], string> extends 1
    ? T[K]
    : T[K] | undefined
}

// prettier-ignore
type StrictPartialFromStrict<ST extends object> =
  | MapUndefinedToOptional<MapNonLiteralToIncludeUndefined<ST>>
  | { [K in Union.Keys<ST>]?: undefined }

type StrictPartial<T extends object> = StrictPartialFromStrict<Union.Strict<T>>

type UpdateSchema = Flatten<StrictPartial<Schema> & { [key: string]: any }>

assertType.equal<
  UpdateSchema,
  | {
      [x: string]: any
      user_id?: number | undefined
      email?: undefined
      auth_token?: undefined
      type: "user"
    }
  | {
      [x: string]: any
      user_id?: undefined
      email?: string | undefined
      auth_token?: string | undefined
      type: "email"
    }
  | {
      [x: string]: any
      type?: undefined
      user_id?: undefined
      email?: undefined
      auth_token?: undefined
    }
>(true)

FYI, assertType is a custom library I use. It shows a typescript warning if the types aren't equal.

[millsp]

I don't think that well find a place for this in the standard library. There you go:

type Schema =
  | { name: string, type: 'user', email: string }
  | { name: string, type: 'team', memberCount: number }

type UpdateType =
  | { name?: string, type: 'user', email?: string, memberCount?: undefined }
  | { name?: string, type: 'team', email?: undefined, memberCount?: number }
  | { name?: string, type?: undefined, email?: undefined, memberCount?: undefined }

type Updater<Schema extends object> =
    // first we create a record that is a schema with all possible properties to which we add `{name?: string}`
    O.Record<U.Exclude<O.Keys<Schema>, 'name'>, never, ['?', 'W']> & {name?: string} extends infer CommonSchema
    ? (
      Schema extends unknown                                      // then we go over the union of Schema so that
      ? O.Required<O.Unionize<CommonSchema & {}, Schema>, 'type'> // each union member of Schema gets
      : never                                                     // gets unionized, fills the CommonSchema
      ) | A.Compute<CommonSchema> // finally we add the empty common schema for passing arbitrary properties
    : never

type t0 = Updater<Schema>                     // it works well
type t1 = O.Diff<Updater<Schema>, UpdateType> // spec compliant

[millsp]

We could write a generic version, but it would not work for your use-case. Still looking into it. Btw, you can use A.Compute instead of Flatten :)

[millsp]

@thesunny I found a real generic solution. This is more elegant, since you can tell which types not to make Optional:

type Schema =
  | { name: string, type: 'user', email: string}
  | { name: string, type: 'team', memberCount: number}

type UpdateType =
  | { name?: string, type: 'user', email?: string, memberCount?: undefined }
  | { name?: string, type: 'team', email?: undefined, memberCount?: number }
  | { name?: string, type?: undefined, email?: undefined, memberCount?: undefined }

type StrictOptional<O extends object, required extends A.Key = never> =
    U.Strict<O> extends infer OS
    ? OS extends unknown
      ? O.Optional<OS & {}, U.Exclude<O.Keys<OS & {}>, required>>
      : never
    : never

type t0 = StrictOptional<Schema | { name: string }, 'type'>
type t1 = O.Diff<StrictOptional<Schema>, UpdateType>

Notice that I had to add { name: string } after schema. And notice also that you can specify fields that must not be made optional.

I called it StrictOptional because I don't use the term Partial.

(cc @tonivj5)

[millsp]

@thesunny do you want me to add it to the standard library?

[millsp]

Maybe this could fit into the Community module.

[thesunny]

Hi @pirix-gh, I will look at it in more depth today.

Thank you for your all your work on ts-toolbelt.

[thesunny]

@pirix-gh after working with it, it feels like the following interface could be simplified:

type t0 = StrictOptional<Schema | { name: string }, "type">

1. We need to know that we have to | the { name: string} in order to not have it in the discriminated portion.
2. Repetitive as we already specified that name is a string { name: string }

If you felt inclined to pursue it, I wonder if this might provide a more intuitive UI.

type t0 = StrictOptional<Schema, "type", "email" | "memberCount">

We specify the Schema, the discriminant and then the discriminated properties. It's generic and intuitive. All the secondary arguments deal with the discriminant which will allow the secondary arguments to be optional. If they are omitted, we can return a type that does not discriminate any properties on the partial.

Feels like such a design would be a good candidate for inclusion.

[millsp]

1) I am not against the second version, but I see a limitation that I am not sure how to overcome cleanly: you see, Schema has 2 union members when the expected UpdateType has 3, this makes the distribution non-generic. Not 1-to-1.

2) This is why I added | { name: string }, so that we can map over 3 members.

3) I don't see any problem with discriminant and discriminated, we would just have to write a modified Strict version that can do that. Or maybe add this feature to Scrict too.

[thesunny]

I may be misunderstanding the issue but can't we do something like:

type StrictOptional<T, Discriminant, Discriminated> =
  | OTHER_PROCESSING_HERE
  |
    Object.Optional<
      {
        [K in Union.Keys<T>]:
          K extends Discriminant ? undefined :
          K extends Discriminated ? undefined :
          T[K]
      },
      Discriminant | Discriminated,
      "flat"
    >

[millsp]

Well, that's partially what I did in https://github.com/pirix-gh/ts-toolbelt/issues/114#issuecomment-630760893

Otherwise it's completely OK that you have your own type for your own scenario, not everything has to or can be generic. The most generic I can get is https://github.com/pirix-gh/ts-toolbelt/issues/114#issuecomment-630774344

And we could improve it with your suggestion type t0 = StrictOptional<Schema, "type", "email" | "memberCount">.

The thing is that you need to add | { name: string } to make your case generic, it's completely normal. To me this this simply means that you are going to allow something that does not have a type and hase some other untracked properties (since you plan to intersect it with [K: string]: any).

In other words, if you want 3 members in the output union, you need 3 members in the input union. Making a 2 member union into a 3 member union is not generic. But maybe I am missing something?

Or maybe it's the fact of adding name by hand that disturbs you? Maybe you would like that to be generic for reasons that are not obvious to me. In this case, we can also do something.

Or, you could just create a type wrapper that hides this away, after all.

[thesunny]

Thank you again for your work.

I think you've taken it farther than you needed to already and I think I'm convinced that it may not be generic enough for inclusion. If somebody else comes in asking for something similar or the same, it could be worth revisiting. :)

[millsp]

Great. As long as your problem got solved and that your software is correctly typed :) I would definitely take a look again if more people need it, and add it to the community.

Writing types is not easy, unfortunately, I have to keep the standard library case-agnostic while trying to satisfy most use-cases... Which leads people to compose their own case-specific types. And when none of this applies, I'm there to help :)

Thank you, you're welcome to ask more questions! Thanks for supporting the project.

Cheers