Typescript: Proposal: covariance and contravariance generic type arguments annotations

[Igorbek]

I have published a proposal document that makes attempt to address an outstanding issue with type variance, that was brought and discussed at #1394

The work is currently not complete, however the idea is understood and just needs proper wording and documenting. I would like to hear feedback from the TypeScript team and community before I waste too much :).

Please see the proposal here - https://github.com/Igorbek/TypeScript-proposals/tree/covariance/covariance, and below is a summary of the idea

Problem

There's a huge hole in the type system that assignability checking does not respect a contravariant nature of function input parameters:

class Base { public a; }
class Derived extends Base { public b; }

function useDerived(derived: Derived) { derived.b; }

const useBase: (base: Base) => void = useDerived; // this must not be allowed
useBase(new Base());    // no compile error, runtime error

Currently, TypeScript considers input parameters _bivariant_.
That's been designed in that way to avoid too strict assignability rules that would make language use much harder. Please see links section for argumentation from TypeScript team.

There're more problematic examples at the original discussion #1394

Proposal summary

Please see proposal document for details.

1. Strengthen input parameters assignability constraints from considering _bivariant_ to considering _contravariant_.
2. Introduce type variance annotations (in and out) in generic type argument positions
   
   
   
   1. in annotates _contravariant_ generic type arguments
   
   
   2. out annotates _covariant_ generic type arguments
   
   
   3. in out and out in annotate _bivariant_ generic type arguments
   
   
   4. generic type arguments without these annotations are considered _invariant_
   
   
3. The annotated generic types annotated with in and out are internally represented by compiler constructed types (transformation rules are defined in the proposal)

Additionally, there're a few optional modifications being proposed:

1. Allow type variance annotation (in and out) in generic type parameter positions to instruct compiler check for co/contravariance violations.
2. Introduce write-only properties (in addition to read-only), so that contravariant counterpart of read-write property could be extracted
3. Improve type inference system to make possible automatically infer type variance from usage

Details

Within a type definitions each type reference position can be considered as:

• _covariant position_, that means for output (such as method/call/construct return types)
• _contravariant position_, that means for input (such as input parameters)

So that when a generic type referenced with annotated type argument, a new type constructed from the original by stripping out any variance incompatibilities:

• write(x: T): void is removed when T referenced with out
• read(): T is reset to read(): {} when T referenced with in
• prop: T becomes readonly prop: T when T referenced with out
• ... see more details in the proposal document

Examples

Say an interface is defined:

interface A<T> {
    getName(): string;  // no generic parameter referenced
    getNameOf(t: T): string;    // reference in input
    whoseName(name: string): T; // reference in output
    copyFrom(a: A<in T>): void;  // explicitly set contravariance
    copyTo(a: A<out T>): void;   // explicitly set covariance
    current: T;         // read-write property, both input and output
}

So that, when it's referenced as A<out T> or with any other annotations, the following types are actually constructed and used:

interface A<in T> {
    getName(): string;  // left untouched
    getNameOf(t: T): string;    // T is in contravariant position, left
    whoseName(name: string): {};   // T is in covariant position, reset to {}
    copyFrom(a: A<in T>): void;  // T is contravariant already
    //copyTo(a: A<out T>): void; // T is covariant, removed
    //current: T;   // T is in bivariant position, write-only could be used if it were supported 
}

interface A<out T> {
    getName(): string;  // left untouched
    //getNameOf(t: T): string;  // T is in contravariant position, removed
    whoseName(name: string): T; // T is in covariant position, left 
    //copyFrom(a: A<in T>): void;  // T is contravariant, removed
    copyTo(a: A<out T>): void;   // T is covariant, left
    readonly current: T;    // readonly property is in covariant position 
}

interface A<in out T> {  // bivariant
    getName(): string;  // left untouched
    //getNameOf(t: T): string;    // T is in contravariant position, removed
    whoseName(name: string): {};   // T is in covariant position, reset to {}
    //copyFrom(a: A<in T>): void;  // T is contravariant, removed
    //copyTo(a: A<out T>): void; // T is covariant, removed
    readonly current: {};    // readonly property is in covariant position, but type is stripped out 
}

Links

• Original suggestion/discussion #1394
• Stricter TypeScript #274
• Suggestion to turn off parameter covariance #6102
  

  Call for people

@ahejlsberg
@RyanCavanaugh
@danquirk

@aleksey-bykov
@isiahmeadows

[isiahmeadows]

This reminds me a _lot_ of Kotlin's covariant and contravariant generics
syntactically. Just a first impression (I haven't really dug deep into this
yet).

On Mon, Sep 5, 2016, 23:23 Igor Oleinikov [email protected] wrote:

I have published a proposal document that makes attempt to address an
outstanding issue with type variance, that was brought and discussed at

1394 https://github.com/Microsoft/TypeScript/issues/1394

The work is currently not complete, however the idea is understood and
just needs proper wording and documenting. I would like to hear feedback
from the TypeScript team and community before I waste too much :).

Please see the proposal here -
https://github.com/Igorbek/TypeScript-proposals/tree/covariance/covariance,
and below is a summary of the idea
Problem

There's a huge hole in the type system that assignability checking does
not respect a contravariant nature of function input parameters:

class Base { public a; }class Derived extends Base { public b; }
function useDerived(derived: Derived) { derived.b; }
const useBase: (base: Base) => void = useDerived; // this must not be allowed
useBase(new Base()); // no compile error, runtime error

Currently, TypeScript considers input parameters _bivariant_
https://github.com/Microsoft/TypeScript-Handbook/blob/master/pages/Type%20Compatibility.md#function-parameter-bivariance
.
That's been designed in that way to avoid too strict assignability rules
that would make language use much harder. Please see links section
<#m_-8167296998011622822_links> for argumentation from TypeScript team.

There're more problematic examples at the original discussion #1394
https://github.com/Microsoft/TypeScript/issues/1394
Proposal summary

Please see proposal document
https://github.com/Igorbek/TypeScript-proposals/blob/covariance/covariance/proposal.md
for details.

1. Strengthen input parameters assignability constraints from
   considering _bivariant_ to considering _contravariant_.
2. Introduce type variance annotations (in and out) in generic type
   argument positions
   
   
   
   1. in annotates _contravariant_ generic type arguments
   
   
   2. out annotates _covariant_ generic type arguments
   
   
   3. in out and out in annotate _bivariant_ generic type arguments
   
   
   4. generic type arguments without these annotations are considered
      
      _invariant_
   
   
   5. The annotated generic types annotated with in and out are
      
      internally represented by compiler constructed types (transformation rules
      
      are defined in the proposal)
   
   

Additionally, there're a few _optional_ modifications being proposed:

1. Allow type variance annotation (in and out) in generic type
   parameter positions to instruct compiler check for co/contravariance
   violations.
2. Introduce write-only properties (in addition to read-only), so that
   contravariant counterpart of read-write property could be extracted
3. Improve type inference system to make possible automatically infer
   type variance from usage

Details

Within a type definitions each type reference position can be considered
as:

• _covariant position_, that means for output (such as
  method/call/construct return types)
• _contravariant position_, that means for input (such as input
  parameters)

So that when a generic type referenced with annotated type argument, a new
type constructed from the original by stripping out any variance
incompatibilities:

• write(x: T): void is removed when T referenced with out
• read(): T is reset to read(): {} when T referenced with in
• prop: T becomes readonly prop: T when T referenced with out
• ... see more details in the proposal document

Examples

Say an interface is defined:

interface A {
getName(): string; // no generic parameter referenced
getNameOf(t: T): string; // reference in input
whoseName(name: string): T; // reference in output
copyFrom(a: A): void; // explicitly set contravariance
copyTo(a: A): void; // explicitly set covariance
current: T; // read-write property, both input and output
}

So that, when it's referenced as A or with any other annotations,
the following types are actually constructed and used:

interface A {
getName(): string; // left untouched
getNameOf(t: T): string; // T is in contravariant position, left
whoseName(name: string): {}; // T is in covariant position, reset to {}
copyFrom(a: A): void; // T is contravariant already
//copyTo(a: A): void; // T is covariant, removed
//current: T; // T is in bivariant position, write-only could be used if it were supported
}
interface A {
getName(): string; // left untouched
//getNameOf(t: T): string; // T is in contravariant position, removed
whoseName(name: string): T; // T is in covariant position, left
//copyFrom(a: A): void; // T is contravariant, removed
copyTo(a: A): void; // T is covariant, left
readonly current: T; // readonly property is in covariant position
}
interface A { // bivariant
getName(): string; // left untouched
//getNameOf(t: T): string; // T is in contravariant position, removed
whoseName(name: string): {}; // T is in covariant position, reset to {}
//copyFrom(a: A): void; // T is contravariant, removed
//copyTo(a: A): void; // T is covariant, removed
readonly current: {}; // readonly property is in covariant position, but type is stripped out
}

Links

• Original suggestion/discussion #1394
  https://github.com/Microsoft/TypeScript/issues/1394
• Stricter TypeScript #274
  https://github.com/Microsoft/TypeScript/issues/274
• Suggestion to turn off parameter covariance #6102
  https://github.com/Microsoft/TypeScript/issues/6102

Call for people

@ahejlsberg https://github.com/ahejlsberg
@RyanCavanaugh https://github.com/RyanCavanaugh
@danquirk https://github.com/danquirk

@aleksey-bykov https://github.com/aleksey-bykov
@isiahmeadows https://github.com/isiahmeadows

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/Microsoft/TypeScript/issues/10717, or mute the thread
https://github.com/notifications/unsubscribe-auth/AERrBPCIezFYgMMrjMLcPWA5UDWKa9hZks5qnNy-gaJpZM4J1bdK
.

[DanielRosenwasser]

@isiahmeadows yup, it's called use-site variance. Kotlin has both use- and declaration- site variance. Check out their paper on it.

I can't say whether we're committed to variance, but I highly suspect that given the way Array is used, use-site variance would be necessary; that's at least my at-a-glance opinion. The bigger problem as I see it is how variance would be inferred. Clearly you wouldn't want to make people write out types more often for this, so inferring would require some more machinery to work well.

[Igorbek]

In my proposal I'm also pointing declaration-site variance as optional part. It must just instruct compiler to verify for variance violations in type definitions, such as you cannot have covariant type taken in contravariant position.

[Igorbek]

@DanielRosenwasser if the inferring is too complicated, it might be a work for tooling as a first stage. Type argument without in and out is naturally understood as an invariant, so if we infer variance, we'd need to have a way to specify invariants explicitly (inv, invariant, exact).

[aaronla-ms]

Please fight for this feature!

One very serious limitation of the assignable to-or-from rule in 3.11.2 is that Promises are unsafe. Consider the following code

var p: Promise<number> = ... ;
var p2: Promise<{}>;
var p3; Promise<string>;
p2 = p;
p3 = p2;
p3.then(s => s.substr(2)); // runtime error, no method 'substr' on number

This code shouldn't be allowed to typecheck; however, this code has passed typechecking in every version of Typescript from 0.8 through 2.0, even with all strictness checks enabled. Assigning {} to string would not typecheck. However, because of function arguments bivariance, the compiler allows Promise<{}> to be assigned to Promise<string>.

Async programming is hard enough without the compiler letting type errors slip through :-)

[isiahmeadows]

Yep. That's a good reason to need it.

Oh, and given the above, I think the default behavior should be changed
after this gets implemented.

On Tue, Sep 13, 2016, 11:47 Aaron Lahman [email protected] wrote:

Please fight for this feature!

One very serious limitation of the assignable to-or-from rule in 3.11.2 is
that Promises are unsafe. Consider the following code

var p: Promise = ... ;
var p2: Promise<{}>;
var p3; Promise;
p2 = p;
p3 = p2;
p3.then(s => s.substr(2)); // runtime error, no method 'substr' on number

This code shouldn't typecheck. Assigning {} to string would not
typecheck. However, because of function arguments bivariance, Promise<{}>
assigns to Promise.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/Microsoft/TypeScript/issues/10717#issuecomment-246726793,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AERrBD2yh8h_mK90nMZPDCV8cX0Wo6cfks5qpsWdgaJpZM4J1bdK
.

[aaronla-ms]

@isiahmeadows I doubt the default will get changed. I messaged the Typescript team back in 2013 about this -- back in 2013, promises were still pretty rare things. They agreed that promises would be important, but for every example I could show that broke, they could find 100 examples of jQuery and other frameworks that would have to forego all type checking if they changed the default. I could tell it was a hard decision for them, but back then jQuery trumped promises, and I think if we're honest, most Typescript users in 2013 would have agreed with that decision.

However, a lot has changed since 2013. Typescript 2.0 has shiny new type checker options. Maybe there's room in a future release to add an option for soundness.

[isiahmeadows]

Good point. And of course, if you want to change the default, you have to
put an option there first (they added one for --strictNullChecks)

On Tue, Sep 13, 2016, 12:07 Aaron Lahman [email protected] wrote:

@isiahmeadows https://github.com/isiahmeadows I doubt the default will
get changed. I messaged the Typescript team back in 2013 about this -- back
in 2013, promises were still pretty rare things. They agreed that promises
would be important, but for every example I could show that broke, they
could find 100 examples of jQuery and other frameworks that would have to
forego all type checking if they changed the default.

However, a lot has changed since 2013. Typescript 2.0 has shiny new type
checker options. Maybe there's room in a future release to add an option
for soundness.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/Microsoft/TypeScript/issues/10717#issuecomment-246732130,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AERrBOnTJXbgq_Whb3T2kX-wdNhP76rkks5qpsj2gaJpZM4J1bdK
.

[Igorbek]

@aaronla-ms @isiahmeadows appreciate your support guys! I'm definitely gonna fight for this.

For me it seems pretty clear that current workaround where parameters are bivariant is not playing well with type safety. However I understand the TypeScript team when they're saying about complexity that could be introduced if just enforce users to write annotations always. So to keep language usage simple, inferring system must be smart enough so that means its implementation can really be challenging.
Probably we could investigate when declaration-site annotations (which is cheap) and simple inference rules solve majority of real-world issues and where use-site annotations would be really necessary.

Issue with promises could be solved with declaration-site variance. Let's imaging how would Promise definition look like:

interface Promise<out T> { // declaration-site covariance that enforces interface verification
    then<TResult>(onfulfilled?: (value: T) => TResult | PromiseLike<TResult>, onrejected?: (reason: any) => TResult | PromiseLike<TResult>): PromiseLike<TResult>;
    then<TResult>(onfulfilled?: (value: T) => TResult | PromiseLike<TResult>, onrejected?: (reason: any) => void): PromiseLike<TResult>;
}

var p: Promise</*implicitly out */ number> = ... ;
var p2: Promise<{}>;
var p3; Promise<string>;
p2 = p; // ok
p3 = p2; // here an error would be cought
p3.then(s => s.substr(2)); // runtime error, no method 'substr' on number

[aaronla-ms]

Issue with promises could be solved with declaration-site variance

I'm assuming you mean with declaration-site variance _and_ parameter contravariance. Otherwise it would still find that Promise<string> satisfies the interface { then: (onfulfilled: (value: {}) => ... } and permit the assignment, right?

[Igorbek]

I'm assuming you mean with declaration-site variance and parameter contravariance

Yes, that is exactly what I meant.

[isiahmeadows]

TypeScript, if I understand correctly, already has parameter contravariance
support (U extends T where T is a class type parameter), which Promises
need. They need the covariant U super T for the other direction, though,
for completeness.

On Tue, Sep 13, 2016, 16:47 Igor Oleinikov [email protected] wrote:

I'm assuming you mean with declaration-site variance and parameter
contravariance

Yes, that is exactly what I meant.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/Microsoft/TypeScript/issues/10717#issuecomment-246818400,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AERrBDxCLqg7FTut9bYh4BGkQvlaQS4zks5qpwvFgaJpZM4J1bdK
.

[yortus]

The current Promise definition allows assigning across types even without the bivariance issue. See #9953 and #10524.

The following simplification of @aaronla-ms's example still compiles fine (without bivariance):

var p: Promise<number> = ... ;
var p3: Promise<string>;
p3 = p;
p3.then(s => s.substr(2)); // runtime error, no method 'substr' on number

[Igorbek]

@isiahmeadows that is not quite the same. you're pointing to type argument constrains (super constraint suggestion is tracked in #7004 and #7265) which are orthogonal to type variance. Constraints set relations between different type parameters (say <T, U> where T extends U). Variance sets relations between same type parameters in variations of produced existential (with concrete type arguments) types (when T is subtype of U then X<out T> is subtype of X<out U>).

[Igorbek]

@yortus that issue is caused by exact same issue - parameter bivariance. If promise had a property of type T (likewise C#'s Task<T>.Result) then it would be covariant. But since Promise exposes its underlying type T within parameter position it becomes bivariant.

[yortus]

@Igorbek I think it's a separate issue. If you comment out the nullary then() overload in the Promise class declaration, then promises of _unrelated_ types can no longer be cross-assigned, and indeed the example will fail with Type 'number' is not assignable to type 'string'.

The bivariance issue remains for subtype/supertypes however.

[Igorbek]

_(fixed the post, it was cut somehow)_
@yortus I'd argue. Currently, lib.d.ts defines only PromiseLike as:

interface PromiseLike<T> {
    /**
    * Attaches callbacks for the resolution and/or rejection of the Promise.
    * @param onfulfilled The callback to execute when the Promise is resolved.
    * @param onrejected The callback to execute when the Promise is rejected.
    * @returns A Promise for the completion of which ever callback is executed.
    */
    then<TResult>(onfulfilled?: (value: T) => TResult | PromiseLike<TResult>, onrejected?: (reason: any) => TResult | PromiseLike<TResult>): PromiseLike<TResult>;
    then<TResult>(onfulfilled?: (value: T) => TResult | PromiseLike<TResult>, onrejected?: (reason: any) => void): PromiseLike<TResult>;
}

This definition prevents assigning PromiseLike<string> to Promise<number>. You can check it produces compiler error. However it can be worked around by using bivariance and intermediate type:

interface P<T> { // this is PromiseLike<P> for simplicity
    then<TResult>(onfulfilled?: (value: T) => TResult | P<TResult>, onrejected?: (reason: any) => TResult | P<TResult>): P<TResult>;
    then<TResult>(onfulfilled?: (value: T) => TResult | P<TResult>, onrejected?: (reason: any) => void): P<TResult>;
}

let p1: P<string>;
let p2: P<number> = p1;  // compiler error (with current compiler)

let p3: P<{ a; }>;
let p4: P<{ b; }> = p3;  // compiler error, again
// but, if I do
let p5: P<{ a; b; }> = p3; // ok, contravariance 
let p6: P<{ b; }> = p5; // ok (P<{ a; }> assigned), covariance (so that is bivariance in total)

[yortus]

@Igorbek that's all true for PromiseLike, but Promise has a nullary then overload which gets used to determine that promises with unrelated types are always structurally compatible. See https://github.com/Microsoft/TypeScript/issues/10524#issuecomment-245120857. There an open PR to fix this, then there will be just the bivariance issue left....

[Igorbek]

ah, ok, that makes sense. I didn't count that is other Promise. Anyway, it's buggy since being covariant by definition, it's bivariant according to type checker.

[aaronla-ms]

Btw, I remembered an old hack abusing property covariance that the Typescript folks showed me first time I hit this issue. Until you have covariance annotations, you could add a dummy optional field of type T. Properties and return values are already covariant (contravariant assignments disallowed), causing your generic to become covariant in T as well.

interface Animal {}
interface Dog extends Animal { woof();}
interface Promise2<T> {
    _covariant?: T; // never actually initialized; just for type checking
    then<U>(cb: (value: T) => Promise2<U>): Promise2<U>;    
}
var p2a: Promise2<Animal> = null;
var p2d: Promise2<Dog> = null;

p2a = p2d; // as desired, is sound and compiler accepts.
p2d = p2a; // as desired, is unsound and compiler rejects: "Type 'Promise2<Animal>' not assignable to 'Promise2<Dog>'. Type 'Animal' not assignable to 'Dog'."

function test3<T, U extends T>(b: Promise2<T>, d: Promise2<U>) {
    b = d; // as desired, is sound and compiler accepts.
    d = b; // as desired, is sound and compiler rejects: "Type 'Promise2<T>' not assignable to 'Promise2<U>'. Type 'T' not assignable to 'U'."
}

Is there an obvious way to extend this to contravariant type as well (e.g. interface IObserver<T> { observe(value: T): void; })?

[Igorbek]

@aaronla-ms nice trick, that technique is also used for emulating nominal types by introducing a private "brand" property.
Unfortunately, TypeScript only uses covariance and bivariance and no contravariance. I don't think there's a way to work this around.
@DanielRosenwasser do you think that the team would at least start discussion about this feature. Can we expect that it would be brought to a slog at some point and when if so?

[Igorbek]

ref #11943 for tracking a good variance-related call to be addressed in the proposal.

[schotime]

Has this progressed any further?

[Igorbek]

I haven't seen any progress for a while. 😞

[RyanCavanaugh]

It's still on our radar, especially as it applies to Promises.

Basically we're struggling with the fact that it greatly increases the cognitive load of the type system (for everyone) in exchange for fixing a few unsoundnesses. If it weren't for Promise being badly affected by this, it'd probably be in the "not worth it" category (e.g. simple function bivariance errors, which is the main symptom, are actually quite rare), but Promise is a key type and it's only getting more important as async becomes mainstream.

[aaronla-ms]

I appreciate the magnitude of the impact, but I'm having some trouble reconciling "greatly increases cognitive load" with "few unsoundnesses". If the unsoundness were really that rare, then the cognitive load would be as well.

(I like to think contravariance is hard for folks because recycling is hard. It's easy to mistake a recycling bin for a combined refuse bin, much like it's easy to mistake a fulfills-promise-of-irecyclable callback for a fulfills-promise-of-{} callback, and accidentally throw a BananaPeel into the the recycling bin)

[RyanCavanaugh]

What I mean is that in exchange for thinking about co- and contravariance all the time (because you need to apply those modifiers correctly, always, everywhere, for this to work), you get less unsoundess in a few corner cases (minus Promises which are no longer in the corner).

[whymarrh]

(minus Promises which are no longer in the corner)

Just jumping in real quick, what do you mean by that?

[goloveychuk]

sorry for offtop, is there any method to globally replace Promise interface with

 interface Promise<T> {
   __promiseBrand: T 
 ....
}

which will also work with async functions?

[aaronla-ms]

@RyanCavanaugh Fair enough. But that still leaves inference and off-by-default on the table. I hear that nullability is being considered as an opt-in type system option, where "let x: number = null" is okay by default, but verboten in strict mode. Maybe bivariance makes sense in non-strict mode, and proper variance in non-strict.

Could even consider inferring it in the non-tricky cases, no?

e.g. interface P<T> { then((value: T) => void): void } is pretty obvious.

[dmikis]

@RyanCavanaugh

Basically we're struggling with the fact that it greatly increases the cognitive load of the type system (for everyone) in exchange for fixing a few unsoundnesses.

Where as this is understandable concern, for me the current situation doesn't seem to be free of cognitive load too. Since type system doesn't provide strong enough guarantees I need always look at code I use twice to make sure that I'm not doing something that will break at some point in runtime.

[ahejlsberg]

See #15104 for a targeted change that makes Promise<T> and similar types properly covariant.

[westonpace]

https://github.com/Microsoft/TypeScript/pull/15104 helps with assignment but it doesn't really fix Promise entirely does it?

let promise: Promise<Animal> = null;
let castedPromise: Promise<Cat> = promise; // Errors now as expected
promise.then((cat: Cat) => { //Should error here but does not
  cat.meow();
});

[zpdDG4gta8XKpMCd]

remove type annotation, instead of (cat: Cat) => do cat =>

[westonpace]

@aleksey-bykov Regrettably that will not work for me. I am building a bunch of action processing chains, similar to express' middleware. Each step requires certain fields to be present and may add fields to the request. For example...

  .then(addParameters)
  .then(addCallingUser)
  .then(addUserRoles)
  .then(doThing);

The definition for addCallingUser looks like

function addCallingUser<T extends HasParameters>(request: T): T & HasCallingUser;

I have about 20-30 of these right now and expect the number to grow. With the bivariance the way it is I can add them to the action in any order I want and it will compile. In reality though it will only work if (for example) addCallingUser comes after addParameters. I can't just leave off the type because the functions are defined externally with no context and it would just be an implicit any in that case.

[schotime]

I am going to have this problem too @westonpace .
Thoughts @ahejlsberg ?

[westonpace]

Also, would super constraints (ala https://github.com/Microsoft/TypeScript/issues/7265) solve this issue by allowing us to specify an additional non-bivariant type? One could define a Promise<T>'s then method as...

    then<V super T, TResult>(onfulfilled: (value: V) => TResult | PromiseLike<TResult>, onrejected?: ((reason: any) => TResult | PromiseLike<TResult>) | undefined | null): Promise<TResult>;

Or would V still be treated bivariantly?

[AlexGalays]

So Promise was fixed in an ad-hoc fashion, but we still have to deal with unsound libraries :p (Observables, streams, etc)

[isiahmeadows]

Here's another issue: functions are way off. Flow nails them, though. (Note the instructive comment on how the errors or lack thereof should be interpreted.)

As for variance, it is possible to calculate without explicit annotations - Haskell, OCaml, and friends have been using implicit variance for the whole time, and in particular, OCaml also has a structural type system.

[gcnew]

@isiahmeadows Have you tried the new --strictFunctionTypes flag? For the given example, it reports the expected errors just as Flow does.

[isiahmeadows]

It'd be easier to do that if the web interface supported that option, though.

[SlurpTheo]

@isiahmeadows What timing! You ask for it and --> same day delivery 😃

([email protected] released OCT 31)

[RastislavMirek]

Is this still activelly discussed? Will we ever get it?

[pelotom]

@RatislavMirek why is it needed now with implicit variance tracking?

[mscharley]

Silly example to show why this is still needed:

interface Queue<in T> {
  add(item: T);
  processItem(item: T);
}

class QueueImpl implements Queue<number> {
  // implementation here which relies on items being numbers.
}

const queue: Queue<unknown> = new QueueImpl();
queue.add("Boom!");

[ahejlsberg]

@mscharley Variance annotations are not needed. TypeScript's structural type system causes variance errors to automatically emerge from whether particular type parameters are used in output positions (covariant), input positions (contravariant), or both (invariant). The real reason you don't get an error in your example above is that members declared using method syntax are compared _bivariantly_. See #18654 for more details, particularly:

The stricter checking applies to all function types, except those originating in method or construcor declarations. Methods are excluded specifically to ensure generic classes and interfaces (such as Array) continue to mostly relate covariantly. The impact of strictly checking methods would be a much bigger breaking change as a large number of generic types would become invariant (even so, we may continue to explore this stricter mode).

If you rewrite your example to use function type syntax you do indeed get an error:

interface Queue<T> {
  add: (item: T) => void;
  processItem: (item: T) => void;
}

class QueueImpl implements Queue<number> {
  add(item: number) {}
  processItem(item: number) {}
}

const queue: Queue<unknown> = new QueueImpl();  // ERROR
queue.add("Boom!");

The reported error is

Type 'QueueImpl' is not assignable to type 'Queue<unknown>'.
  Types of property 'add' are incompatible.
    Type '(item: number) => void' is not assignable to type '(item: unknown) => void'.
      Types of parameters 'item' and 'item' are incompatible.
        Type 'unknown' is not assignable to type 'number'.

which explains why your contravariant usage of the type parameter prohibits the assignment.

[mscharley]

Thank you for the example of how I can fix my code.

Frankly, the fact those two definitions aren’t equivalent is more confusing than any introduction of explicit variance, in my opinion.

I understand the need to support both syntaxes but they really should be equivalent since they’re describing the same shape: a function on an object that takes a T and returns nothing. Especially for interfaces, which don’t need to implemented by a class and can be implemented with just a regular object.

[Igorbek]

@ahejlsberg thank you for stepping into here again. It has been indeed a great progress in this area so far and majority of the related issues were addressed nicely. However I do see value of having use-site variance annotations.

Here's the example that is still problematic:

interface Queue<T> {
    enqueue(item: T): void;
    dequeue(): T;
}

interface QueueProcessor<T> {
    // there's nothing that tells how is the type T going to be used
    process(queue: Queue<T>): void;
}

class DogReader implements QueueProcessor<Dog> {
    process(queue: Queue<Dog>) { queue.dequeue().woof(); }
}

class CatWriter implements QueueProcessor<Animal> {
    process(queue: Queue<Animal>) { queue.enqueue(new Cat); }
}

declare const baseQueue: Queue<Animal>;
declare const deriveQueue: Queue<Dog>;

const baseProcessor: QueueProcessor<Animal> = new DogReader; // ok, should have been an error
const derivedInitializer: QueueProcessor<Dog> = new CatWriter; // ok, should have been an error

baseQueue.enqueue(new Animal); baseProcessor.process(baseQueue);   // boom
derivedInitializer.process(deriveQueue); deriveQueue.dequeue().woof(); // boom

So, regardless TS cannot detect variance violation here, we could split type Queue<T> into ReadonlyQueue<T> and WriteonlyQueue<T> and then define ReadonlyQueueProcessor<T> and WriteonlyQueueProcessor<T>. In this case, having stricter rules, we'd detect errors and rewrite DogReader to implement ReadonlyQueueProcessor<T>.

What I was suggesting is to introduce annotations that would simply construct new types that filter out everything that violates variance. So that Queue<out T> would effectively be the same as ReadonlyQueue<T>.

Like so:

interface Queue<T> {
    enqueue(item: T): void;
    dequeue(): T;
}
type Queue<out T> = { dequeue(): T; }
type Queue<in T> = { enqueue(item: T): void; }

interface QueueProcessor<T> {
    process(queue: Queue<T>): void;
}
type QueueProcessor<out T> = { process(queue: Queue<out T>): void; }
type QueueProcessor<in T> = { process(queue: Queue<in T>): void; }

class DogReader implements QueueProcessor<out Dog> { .. } // error if it tries to enqueue
class CatWriter implements QueueProcessor<in Animal> { ... } // error if it tries to dequeue

In my opinion, it would very much simplify read-only story and would make constructing read-only and write-only view to type trivial, like type ReadonlyArray<T> = Array<out T>.
Also note, that it is not just about read/write-only-ness as it can be used to construct any combination of views with respect to their type variance: ItemProcessor<in TIn, out TOut>.
Also note, having type Readonly<T> = { readonly [K in keyof T]: T[K]; }, it is not that Readonly<A<X>> is the same as A<out X>.

[zpdDG4gta8XKpMCd]

i like and hate the way it is, i admire the effort of the design team to put in use every subtle difference, great job team!

[RastislavMirek]

@ahejlsberg @pelotom Implicit variance tracking is definitelly nice progress. However, it can be proven that language that contains generics but not variance cannot be sound (unless it forces invariance everywhere or it does not have inheritance which is not the case with TS). See for example comment by @Igorbek above.

I understand that soundness is not the ultimate north star for TS but still, catching as many type-related issues as possible would be great. Explicit variance is no evil. It is not hard-to-use. It is part of almost every modern language that offers generics including the most popular ones. Today, people are used to it, understand it and expect it. So why not simplify their lifes and just give it to them?

[isiahmeadows]

@RastislavMirek Borderline off-topic, but do you have any links to any papers showing this? I'm just asking out of curiosity.

[Jessidhia]

The main place where I encounter the issue is with "out" parameters, the most common being a React ref.

At the time this issue was made, React didn't use createRef or useRef so the problem was narrowly avoided with the use of the bivarianceHack trick for callback refs.

But ref objects are just { readonly current: T | null } (readonly from the point of view of the user; React will write into it). This is also normally not a problem when the ref object that you create is fully under your control, but it becomes troublesome when trying to communicate between components with ref.

The easiest example is that <a ref={React.createRef<HTMLElement>()} /> will be a type error because TS will think we're trying to give it an HTMLElement when it wants HTMLAnchorElement; but in reality it's the opposite. We're asking for an HTMLElement, and if it gives us an HTMLAnchorElement we'll be just as fine with it.

A heuristic I'd thought of proposing is that if all the keys of the object that you give are readonly but the argument type expects them to be writable, that this object should be considered contravariant instead; but that can cause problems with unsoundly-written type definitions, particularly ones that don't correctly annotate their ReadonlyArray.

[Igorbek]

Right, in React ref would be used like this:

type Ref<T> = { current: T | null; }

// React writes ref
function _reactSetRef<T>(ref: Ref<in T>, value: T | null) {
 //                               ^^^^ allow anything that T is assignable to
  ref.current = value; // ok
}

const myref = createRef<HTMLElement>();

// Props<'input'> = { ... ref: Ref<in HTMLInputElement> }
<input ref={myref} />; // ok

[isiahmeadows]

React's contravariant ref could be solved just by making that property write-only, if only that existed.

[Adjective-Object]

I think I'm bumping into this when using slotted components in react?

Say for example I have some component

<HostComponent<T> renderView={ViewRenderer<T>} />

where renderView is used to render some subcomponent of HostComponent

<HostComponent<Chicken> renderView={BirdView} />

My instinct is that this should typecheck since ViewRenderer only _reads_ T to render a component, but it doesn't in current typescript since ViewRenderer<T> is covariant on T instead of contravariant.

---

Edit: for anyone else looking at a similar problem, the solution I came to was to reassign the identifiers through a utility type. I'm not happy with the solution, but at least it'll throw an error at the callsite if the types change in the future.

/**
 * Because typescript doesn't support contravariance or writeonly props to components,
 * 'write-only' parameter (e.g. generic component slots) must be cast to exact types at
 * the callsite.
 *
 * See related typescript issue
 * https://github.com/Microsoft/TypeScript/issues/10717
 *
 * This alias checks that the type we're casting to is a subtype of the exact expected type
 * so the cast site won't break silently in the future.
 */
type VarianceHack<ParentType, ChildType> = ChildType extends ParentType ? ParentType : never;
const ChickenView = BirdView as VarianceHack<
    ViewRenderer<Chicken>,
    typeof BirdView
>;

///...

<HostComponent<Chicken> renderView={ChickenView} />

[RyanCavanaugh]

I came back to this issue because we've been seeing a lot of confusion around how variance gets measured. A particular example is something that looks covariant:

class Reader<T> {
    value!: T;
    getProperty(k: keyof T): T[keyof T] {
        return this.value[k];
    }
}

type A = { a: string };
type AB = A & { b: number };

function fn<T>(inst: Reader<A>) {
    const s: string = inst.getProperty("a");
}

declare const ab: Reader<AB>;
// Disallowed, why?
fn(ab);

It really seems like Reader<T> is covariant over T, and it is as long as k never originates in an aliased keyof T position. You have to add a field somewhere but it doesn't modify the class variance:

class Reader<T> {
    value!: T;
    someKey!: keyof T;

    getProperty(k: keyof T): T[keyof T] {
        return this.value[k];
    }
}

type A = { a: string };
type AB = A & { b: number };

function fn<T>(inst: Reader<A>) {
    const s: string = inst.getProperty(inst.someKey);
}

declare const ab: Reader<AB>;
// Legal
ab.someKey = "b";
// Causes s: string to get a number
fn(ab);

Indeed if you just extracted out getProperty to a bare function, it'd be obviously contravariant:

declare const a: A;
declare const kab: keyof AB;
declare function getProperty<T, K extends keyof T>(value: T, key: K): T[K];
// Illegal because kab could be 'b'
getProperty(a, kab);

The problem is the original example here is not really contravariant/invariant without some aliasing step, and you have no way to assert that this aliasing doesn't actually occur in your program - the measured variance is the measured variance, full stop.

The follow-on is that it's not clear what to do. If you let you write

class Reader<covariant T> {

we'd presumably just have to error on the declaration of getProperty, because it really is not a covariant usage - in any reasonable definition, this would work the same way implements does (an assertion of a measured fact, not an override of reality).

It seems like what you want is some way to annotate specific use sites of T to override their variance measurement, or maybe some crazy way to define getProperty in a way that it disallows aliased values of k, though it's not clear how that's even remotely possible.

As for use-site variance annotations, I don't think this is a good route. The variance of any particular site is purely manifested by its position; the only real missing feature here is writeonly (which I think is ultimately inevitable, despite our protests).

[Igorbek]

@RyanCavanaugh thank you for getting back to this issue and keeping thinking of attacking it in some direction.

I totally agree that this a common confusion what is variance is and how it relates to readonly-ness. It seems to me that most of the time covariance and readonly-ness are used interchangeably.

However, I would argue that in my original proposal I had the same misunderstanding and, more importantly, I still insist that use-site variance has its own dedicated value for the type system correctness and expressiveness.

First, I want to admit that having such a level of expressiveness definitely requires a very advanced understanding of it is and how to use it correctly. It is supposed to be used by library and type definition authors. Therefore, I agree the feature needs to be designed very carefully to not harm most regular users. No rush at all, especially many use-cases were already addressed with readonly and stricter variance checking.

As for use-site variance annotations, I don't think this is a good route. The variance of any particular site is purely manifested by its position; <...>

There're still positions where the variance in respect to generic types cannot be manifested by its position. A simple example can show this:

/** Moves data from the source array to the destination array and return the destination's final size */
function moveData<T>(source: readonly T[], destination: writeonly T[]): number {
  while (source.length) {
    destination.push(source.pop()); // 'pop' is not defined on ReadonlyArray<T>
  }
  return destination.length; // 'length' getter is not defined on WriteonlyArray<T>
}

What the intent there really is not readonly/writeonly for these arrays. Is that a guarantee that only subtypes of T will be used from source and supertypes of T in destination:

moveData(cats, animals); // allowed
moveData(animals, cats); // disallowed

The correct signature to express that would be:

declare function moveData<T>(
  source: { readonly length: number; pop(): T; }, // T is covariant
  destination: { readonly length: number; push(item: T): void; } // T is contravariant
): number;

So, what I've been suggesting, is to allow use-site variance annotations that construct new types from existing:

declare function moveData<T>(
  source: out T[], // not the same as readonly T[]
  destination: in T[] // not the same as writeonly T[]
): number;

The examples with arrays usually are very confusing because they are usually really meant readonly/writeonly. But I wanted to show with simple constructs.

[suchipi]

Just hit the exact case @Jessidhia mentioned in https://github.com/microsoft/TypeScript/issues/10717#issuecomment-461704350.

Is there a workaround other than // @ts-ignore available?