There are times when users need to express that a type might exist depending on the environment in which code will eventually be run. Typically, the intent is that if such a type can be manufactured, a library can support operations on that type.
One common example of this might be the Buffer
type built into Node.js. A library that operates in both browser and Node.js contexts can state that it handles a Buffer
if given one, but the capabilities of Buffer
aren't important to the declarations.
export declare function printStuff(str: string): void;
/**
* NOTE: Only works in Node.js
*/
export declare function printStuff(buff: Buffer): void;
One technique to get around this is to "forward declare" Buffer
with an empty interface in the global scope which can later be merged.
declare global {
interface Buffer {}
}
export declare function printStuff(str: string): void;
/**
* NOTE: Only works in Node.js
*/
export declare function printStuff(buff: Buffer): void;
For consuming implementations, a user might need to say that a type not only exists, but also supports some operations. To do so, it can add those members appropriately, and as long as they are identical, they will merge correctly. For example, imagine a library that can specially operate on HTML DOM nodes.
function printStuff(node: HTMLElement) {
console.log(node.innerText);
}
A user might be running in Node.js or might be running with "lib": ["dom"]
, so our implementation can forward-declare HTMLElement
, while also declaring that it contains innerText
.
declare global {
interface HTMLElement {
innerText: string;
}
}
export function printStuff(node: HTMLElement) {
console.log(node.innerText);
}
Using interface merging works okay, but it has some problems.
Interface merging doesn't always correctly resolve conflicts between declarations in two interfaces. For example, imagine two declarations of Buffer
that merge, where a function that takes a Buffer
expects it to have a toString
property.
If both versions of toString
are declared as a method, the two appear as overloads which is slightly undesirable.
declare global {
interface Buffer {
// We only need 'toString'
toString(): string;
}
}
export function printStuff(buff: Buffer) {
console.log(buff.toString());
}
////
// in @types/node/index.d.ts
////
interface Buffer {
toString(encoding?: string, start?: number, end?: number): string;
}
Alternatively, if any declaration of toString
is a simple property declaration, then all other declarations will be considered collisions which will cause errors.
declare global {
interface Buffer {
toString(): string
}
}
////
// in @types/node/index.d.ts
////
interface Buffer {
toString: (encoding?: string, start?: number, end?: number) => string;
}
The former is somewhat undesirable, and the latter is unacceptable.
Another problem with the trick of using interfaces for forward declarations is that it only works for classes and interfaces. It doesn't work for, say, type aliases of union types. It's important to consider this because it means that the forward-declaration-with-an-interface trick breaks as soon as you need to convert an interface to a union type. For example, we've been taking steps recently to convert IteratorResult
to a union type.
An empty interface declaration like
interface Buffer {}
allows assignment from every type except for unknown
, null
, and undefined
, because any other type is assignable to the empty object type ({}
).
Proposed is a new construct intended to declare the existence of a type.
exists type Foo;
A placeholder type declaration acts as a placeholder until a type implementation is available. It provides a type name in the current scope, even when the concrete implementation is unknown. When a non-placeholder declaration is available, all references to that type are resolved to an implementation type.
The example given is relatively simple, but placeholder types can also support constraints and type parameters.
// constraints
exists type Foo extends { hello: string };
// type parameters
exists type Foo<T>;
// both!
exists type Foo<T, U> extends { toString(): string };
A formal grammar might appear as follows.
PlaceholderTypeDeclaration ::
exists
[No LineTerminator here]type
BindingIdentifier TypeParametersopt Constraintopt;
A placeholder type can co-exist with what we might call an implementation type - a type declared using an interface, class, or type alias with the same name as the placeholder type.
In the presence of an implementation type, a placeholder defers to that implementation. In other words, for all uses of a type name that references both a placeholder and an implementation, TypeScript will pretend the placeholder doesn't exist.
A placeholder type is allowed to declare an upper bound, and uses the same syntax as any other type parameter constraint.
exists type Bar extends { hello: string };
This allows implementations to specify the bare-minimum of functionality on a type.
exists type Greeting extends {
hello: string;
}
function greet(msg: Greeting) {
console.log(msg.hello);
}
If a constraint isn't specified, then the upper bound is implicitly unknown
.
When an implementation type is present, the implementation is checked against its constraint to see whether it is compatible. If not, an implementation should issue an error.
exists type Foo extends {
hello: string
};
// works!
type Foo = {
hello: string;
world: number;
};
exists type Bar extends {
hello: string;
}
// error!
type Bar = {
hello: number; // <- wrong implementation of 'hello'
world: number;
}
A placeholder type can specify type parameters. These type parameters specify a minimum type argument count for consumers, and a minimum type parameter count for implementation types - and the two may be different!
For example, it is perfectly valid to specify only type arguments which don't have defaults at use-sites of a placeholder type.
exists type Bar<T, U = number>;
// Acceptable to omit an argument for 'U'.
function foo(x: Bar<string>) {
// ...
}
But an implementation type must declare all type parameters, even default-initialized ones.
exists type Bar<T, U = number>;
// Error!
// The implementation of 'Bar' needs to define a type parameter for 'U',
// and it must also have a default type argument of 'number'.
interface Bar<T> {
// ...
}
Whenever multiple placeholder type or implementation type declarations exist, their type parameter names must be the same.
Different instantiations of placeholders that have type parameters are only related when their type arguments are identical - so for the purposes of variance probing, type parameters are considered invariant unless an implementation is available.
Because placeholder types are just type variables that recall their type arguments, relating placeholders appears to fall out from the existing relationship rules.
The intent is
In effect, two rules in any of our type relationships should cover this:
- S and T are identical types.
- S is a type parameter and the constraint of S is [[related to]] T.
Because different parts of an application may need to individually declare that a type exists, multiple placeholder types of the same name can be declared, and much like interface
declarations, they can "merge" in their declarations.
exists type Beetlejuice;
exists type Beetlejuice;
exists type Beetlejuice;
In the event that multiple placeholder types merge, every corresponding type parameter must be identical. On the other hand, placeholder constraints can all differ.
interface Man { man: any }
interface Bear { bear: any }
interface Pig { pig: any }
exists type ManBearPig extends Man;
exists type ManBearPig extends Bear;
exists type ManBearPig extends Pig;
When multiple placeholder types are declared, their constraints are implicitly intersected to a single upper-bound constraint. In our last example, ManBearPig
's upper bound is effectively Man & Bear & Pig
. In our first example with Beetlejuice
, the upper bound is unknown & unknown & unknown
which is just unknown
.
C and C++ also support forward declarations of types, and is typically used for opaque type handles. The core idea is that you can declare that a type exists, but can never directy hold a value of that type because its shape/size is never known. Instead, you can only deal with pointers to these forward declared types.
struct FileDescriptor;
FileDescriptor* my_open(char* path);
void my_close(FileDescriptor* fd);
This allows APIs to abstract away the shape of forward-declared types entirely, meaning that the size/shape can change. Because these can only be pointers, there isn't much you can do with them at all (unlike this implementation).
Several other programming languages also support some concept of "opaque" or "existential" types, but are generally not used for the same purposes. Java has wildcards in generics, which is typically used to allow one to say only a bit about how a collection can be used (i.e. you can only write Foo
s to some collection, or read Bar
s, or you can do absolutely nothing with the elements themselves). Swift allows return types to be opaque in the return type by specifying that it is returning some SuperType
(meaning some type variable that extends SuperType
).
We have two "obvious" options.
I believe that additive constraints are the more desirable behavior for a user. The idea is that different parts of your application may need different capabilities, and given that interface
s can already model this with interface merging, using intersections provides a similar mechanism.
In part, yes! When no implementation type exists, a placeholder type acts as a bounded existential type variable.
Sorry I'm not sure what you're talking about. Please move along and don't write blog posts about how TypeScript is adding bounded existential types.
function foo() {
exists type Foo;
return null as any as Foo;
}
Maybe! It might be possible to disallow placeholder types from escaping their declaring scope. It might also be reasonable to say that a placeholder can only be declared in the top level of a module or the global scope.
exists
keyword?Maybe we don't need the exists
keyword - I am open to doing so, but wary that we are unnecessarily abusing the same syntax. I'd prefer to be explicit that this is a new concept with separate syntax, but if we did drop the exists
, we would change the grammar to the following.
PlaceholderTypeDeclaration ::
type
[No LineTerminator here] BindingIdentifier TypeParametersopt Constraintopt;
What happens if no implementation type is found? Can we assign anything to a placeholder type reference?
For example:
exists type Buffer extends { toArray(): number[] }
export function printStuff(buff: Buffer) {
console.log(buff.toString());
}
printStufff({ toArray() { return [0] } }); // error here ?
What happens if no implementation type is found? Can we assign anything to a placeholder type reference?
In your example only Buffer
is assignable to Buffer
(unless you have another placeholder type which extends Buffer
. It pretty much acts like a type parameter.
So to answer your question
I'm not a fan of exists
as a keyword, but I suppose it's no worse than declare
. It does seem like you are missing a form for declaring a module exists as well as types in that module, so as not to introduce an ambient module incorrectly:
exists module "net" {
exists type Server;
}
export function connect(srv: import("net").Server) {...}
Alternatively:
exists type Server from "net";
@rbuckton I think you can already do that with module augmentations and ambient module declarations which merge.
// globals.d.ts
declare module "net" {
export exists type Server;
}
// consumer.ts
import net = require("net");
function connect(srv: net.Server): {
// ...
}
Two problems with that:
Just as the goal with exists type X
is not to define but declare a placeholder, I don't want to define "net"
in my example but declare a placeholder.
I don't want to introduce a "net"
module in my declaration file that makes import ... from "net"
work in your project even when you don't have the NodeJS types installed.
Sorry I'm not sure what you're talking about. Please move along and don't write blog posts about how TypeScript is adding bounded existential types.
Aren’t they literally existential types though? You’re introducing an opaque type variable in an otherwise non-generic context, to stand in for any type it needs to be; (my understanding is) existential types do the same, allowing parameterization on a type without T
contributing to the overall type of the function/object/whatever.
Or maybe I’m being r/whoosh’d with this one...
Nevermind, I didn’t read it closely enough. It’s for forward declaration of types that will eventually be properly defined. That’s a different animal entirely.
🏠🚲
Given that exists
implies existential types which these explicitly aren't, and that this is basically just an ambient declaration of a type, why couldn't the syntax simply be:
declare type Bar<T, U = number>;
declare type ManBearPig extends Man;
declare type ManBearPig extends Bear;
declare type ManBearPig extends Pig;
Both of the above declarations are syntax errors today, so we can just appropriate declare
here and avoid the need for a new keyword entirely, right?
@fatcerberus I like that idea of a declare type ...
. Doesn't create a new keyword, and it arguably is more appropriate to use here (and less of an abuse of terminology) than exists
.
There's an issue with not being able to supply the minimum expected definition. Let's say I have the following package "packageA":
// tsconfig.json
{ "compilerOptions": { "target": "esnext", "lib": ["node"] } }
// index.ts
// yes, this is a bad example, but bear with me...
declare global {
declare type Buffer;
}
export function copyBuffer(src: Buffer, dest: Buffer, srcStart: number,
destStart: number, count: number): void {
while (count > 0) {
dest.writeUint8(src.readUint8(srcStart++), destStart++);
count--;
}
}
Now lets say I consume "packageA" from "packageB":
// tsconfig.json
{ "compilerOptions": { "target": "esnext" } }
```ts
import { copyBuffer } from "packageA";
declare global {
interface Buffer { iAmABuffer: boolean }
}
const src: Buffer = { iAmABuffer: true };
const dest: Buffer = { iAmABuffer: true };
copyBuffer(src, dest, 0, 0, 0);
The package "packageA" isn't indicating to "packageB" where `Buffer` should come from or what its definition needs to look like, so as far as "packageB" is concerned, it has properly satisfied the constraints of `Buffer` for "packageA".
By specifying a constraint (`exists type Buffer extends { ... }`), you can at least indicate the minimum implementation necessary. However, you could have just as easily written `copyBuffer` like this:
```ts
export function copyBuffer(
src: { readUint8(offset: number): number },
dest: { writeUint8(value: number, offset: number },
srcStart: number,
destStart: number,
count: number): void { ... }
However this seems highly repetitive and overcomplicated.
At the end of the day, what this boils down to is this:
I have an API I produce that is optional and only usable in certain contexts. If you don't use this API, you don't need to also include the dependencies to satisfy that context.
So its not the case that just any Buffer
will do, but rather "you can't use this particular API if you don't have the types for NodeJS's Buffer
" and "if you don't use this particular API you don't need the types for NodeJS's Buffer
".
In a way, I feel like this makes the whole exists type Buffer from "buffer"
syntax more reliable, as it's not just any Buffer
, but the one from that particular package/module. The syntax for requiring a global would be a bit trickier though. Perhaps what is needed is a package/lib hint for typings...
// just some syntax bikeshedding...
exists type Server from "net" in package "@types/node";
exists type Buffer in package "@types/node";
exists type Promise in lib "es2015.promises";
In these cases, your project wouldn't need to have a dependency on "@types/node" or a "lib": ["es2015.promises"]
in your tsconfig.json if you don't use the APIs from the other package. If you don't have the requisite references, the types are basically never
. If you do have the references, the types light up with the correct typings.
ExistentialTypeDeclaration:
`exists` `type` Identifier TypeArguments? ExistentialFromClause? ExistentialInClause
ExistentialFromClause:
`from` StringLiteral
ExistentialInClause:
`in` `package` StringLiteral
`in` `lib` StringLiteral
I still don’t see why we can’t just use declare
. It’s like every other use of declare
we have already: to say that something exists for which you don’t (yet) have details about because the concrete definition is elsewhere.
I still don’t see why we can’t just use declare. It’s like every other use of declare we have already: to say that something exists for which you don’t (yet) have details about because the concrete definition is elsewhere.
Daniel explicitly calls out why just adding a declare
or a module augmentation for the type is problematic in the Issues section, above. declare global { interface Buffer {} }
"works" today, however that means you now have a Buffer
type that is an empty object, so everything (except null
/undefined
/void
) is assignable to the argument.
Yes, I saw that bit. I specifically meant this:
declare type Buffer;
// instead of exists type Buffer;
Which is currently a syntax error.
This feature reminds me of the weak
symbol. To follow the established precedent, would weak
work instead of exists
?
@patrickroberts thanks for bringing up that example - I hadn't heard of weak symbols but it's conceptually very similar.
The issue with weak
as a keyword is the confusion with another term we've used to describe certain types. Today, an object type which declares only optional properties and no signatures is considered weak, and we do extra checking in the presence of weak types.
@DanielRosenwasser I can't say I'd heard that term describing such objects before. Is that term used just within TypeScript internals, or should that term also have meaning to end users of TypeScript?
It's not widely used, but it's been publicly explained enough, even within our release notes. I'm flexible, but I'd rather avoid weak
unless we feel the other options don't suffice.
This would be a really great addition to the language. I often want to use types and modules instead of classes, but the inability to hide the implementation of the type is annoying. Some examples where this would be useful in the wild are the file descriptor in Node's fs.open
function and a lot of the WebGL types, such as WebGLBuffer
.
Reason and Ocaml have abstract types that are used in exactly this way.
There's some more discussion in #321 which is for a similar request.
EDIT Never mind. I misread the proposal.
The exists
keyword seem to cause some confusion. Since the explanation of this proposal mentions "forward declaration" in several places perhaps it could use a forward
keyword instead of exists
?
forward type Foo;
"Placeholder type" is also mentioned several times, so that might also be something to consider:
placeholder type Foo;
(I may be missing something, I only skimmed the proposal so I don't have a deep understanding of it yet)
For syntax, I'm pretty fine with just
[declare] type Foo;
and
[declare] type Foo extends Whatever;
because it mirrors our other shorthand
declare module "foo";
in which we just take the part of the declaration we do have (the name) and elide the body.
Yes, having studied this proposal a bit deeper, declare
makes a lot of sense since that is already used for the same purpose elsewhere.
Yes, that’s what I’ve been saying all along! :wink:
Just thought I'd post my declaration-merging-as-placeholder-type experiment here.
interface ExpectedConfigT {
x : number,
y : string,
toString () : string,
}
/**
* Your library
*/
interface ConfigT extends ExpectedConfigT {
__doesNotExist? : unknown;
}
declare function getX () : ConfigT["x"];
declare function getToString () : ConfigT["toString"];
/**
* Users of your library
*/
interface ConfigT {
x : 1337,
y : "hello",
toString(encoding?: string, start?: number, end?: number): string,
}
/**
* Type is `1337`
*/
const x = getX();
/**
* Type is `(encoding?: string | undefined, start?: number | undefined, end?: number | undefined) => string`
*/
const toStringFunc = getToString();
Expect exist
instead of exists
typos.
A distraction absent with declare
.
There's a temptation to use declare
, but it's already valid code:
declare type foo = string;
Given code like
exists type Foo;
exists type Bar;
declare let x: Foo;
declare let y: Bar;
what operations are available on x
when a full defn of Foo
/Bar
isn't visible? I saw this comment but I'm not sure what "acts like a type parameter" means -- do you mean they're as if function f<Foo, Bar>() { let x: Foo; }
?
What sort of error messages do you get when you try to use the undefined type? Is x = y
legal?
To Daniel's point about declare type
being existing valid syntax, the existential version should be orthogonal: if there's an =
then it's the old syntax, if not then it's the new one. That still may be more confusion that is warranted, but it's not actually overlapping syntax.
To Evan's question, that's how I read it. They should _not_ be mutually assignable. Indeed, in the templated function case, they're not:
Type 'Bar' is not assignable to type 'Foo'.
'Foo' could be instantiated with an arbitrary type which could be unrelated to 'Bar'.(2322)
Obviously I'd expect a slightly different error message, but something to that effect would suffice.
@evmar @shicks answered pretty well, but I'll try to go deeper here.
what operations are available on
x
when a full defn ofFoo
/Bar
isn't visible?
It depends entirely on the constraints. In your example, Foo
and Bar
are both implicitly bounded by unknown
. This means you can only claim that functions produce and consume them. When you're able to produce a value of type Foo
, you can only pass it to functions that take a Foo
or the constraint of Foo
(unknown
).
placeholder type Foo;
placeholder type Bar;
declare let x: Foo;
// error! Foo is unrelated to Bar
let y: Bar = x;
// okay: Foo's constraint is 'unknown'
let z: unknown = x;
In that sense, it acts just like a type parameter, which is only assignable to itself and its upper bound constraint.
Is
x = y
legal?
in your example, x = y
is not legal because you just don't know what types Foo
and Bar
will be in any given runtime. Only when Foo
and Bar
have an implementation is it plausible for them to be assignable.
If there is some sort of relationship between Foo
and Bar
, you can always constrain one to the other.
placeholder type Foo extends Bar;
placeholder type Bar;
What sort of error messages do you get when you try to use the undefined type?
We could use the existing message for type parameters, but I'd prefer to specialize it.
I wanted to give an update on the status here. As I implemented this and we explored the problem space, we're going to be easing off of placeholder types at least for the near future.
Placeholder types blend a lot of existing concepts into a new language construct:
There was a question of whether some of these concepts could be introduced individually. Ideas there (which we aren't necessarily actively exploring) included:
default
keyword to say that they yield to other types but act a certain way until then.One problem with placeholders is that they aren't able to discuss the values and types of a module that might only partially exist. While this isn't widespread in the module ecosystem right now, that ecosystem is fairly nascent and we don't want to box ourselves into a design that doesn't have a design ready. But we did ask ourselves quite a few times whether it made sense to have a
placeholder module "foo" {
export placeholder type Thing;
}
and we want to understand if there's something more there.
Another problem with placeholders that we realized was the fact that while this feature was meant to give better behavior than the interface merging forward declaration trick. Ironically, the fact that interfaces are written with merging in mind can end up causing issues with this feature.
For example, if you're trying to forward-declare Map<K, V>
with a placeholder type, and someone includes some older-style forward declaration or a future augmentation for Map<K, V>
from es20xx
, that will basically ruin the forward declaration. It's not necessarily a deal-breaker, but that does sound like an annoying problem.
The more we thought about it, it was clear that placeholders gave us a better forward-declaration, but that's only one piece of the broader set of issues caused by needing to compile under different runtime environments. If we were only solving part of the problem, was it worth it?
And further, did this feel better? Only for what people were trying to write today. @andrewbranch brought up a good point which was that forward declarations are often not the way people think about solving the problem. If you know you need a forward declaration, then a placeholder type is great; but most users don't know that they need a forward type declaration.
Placeholder type seems like OCaml functors or ReasonML module functions.
I use keywords namespace
below, but it can be used for other keywords like module
and so on.
// lib
namespace type NeedBuffer {
export placeholder type Buffer extends { toString: (encoding?: string) => string; }
// maybe we can provide some functions for Buffer
export function BufferToString(buff:Buffer) {
return buff.toString()
}
}
namespace makePrintStuff = (needBuffer: NeedBuffer) => {
type PrintStuffFunction =
& (str: string) => void
& (needBuffer.Buffer extends never ? unknown : (buff:needBuffer.Buffer) => void)
export const printStuff: PrintStuffFunction = ......
}
// node env
type NodeBuffer = Buffer;
namespace nodeEnvironment = {
export type Buffer = NodeBuffer
}
namespace nodePrintStuff = makePrintStuff(nodeEnvironment);
exports = nodePrintStuff;
// browser env
namespace browserEnvironment = {
export type Buffer = never
}
namespace browserPrintStuff = makePrintStuff(browserEnvironment);
exports = nodePrintStuff;
But this way of writing is tricky. We should not get information about type Buffer in makePrintStuff.
So I rewrote the above example:
// file ./make-print-stuff.ts
export namespace type ProvideToString {
export placeholder type T
// maybe we can provide some functions for Buffer
export declare function convertToString(value:T):string
}
export namespace makePrintStuff = (provideToString: ProvideToString) => {
type PrintStuffFunction = (value: provideToString.T) => void
export const printStuff: PrintStuffFunction = value => console.log(provideToString.convertToString(value))
}
// file: ./feature-node.ts
import { makePrintStuff } from './make-print-stuff'
namespace nodeEnvironment = {
export type T = Buffer | string;
export const convertToString = (x:T) => typeof x === 'string' ? x : x.toString()
}
namespace nodePrintStuff = makePrintStuff(nodeEnvironment);
exports = nodePrintStuff;
// file: ./feature-browser.ts
import { makePrintStuff } from './make-print-stuff'
namespace browserEnvironment = {
export type T = string;
export const convertToString = (x:T) => x
}
namespace browserPrintStuff = makePrintStuff(browserEnvironment);
exports = nodePrintStuff;
nodejs support Conditional Exports. Library user could provide node
and browser
version implements and makePrintStuff lib.
{
"main": "./feature-node.cjs",
"exports": {
".": {
"import": "./feature-node.mjs",
"require": "./feature-node.cjs",
"default": "./feature-node.cjs",
"browser": "./feature-browser.mjs"
},
"./make-print-stuff": {
"import": "./make-print-stuff.mjs",
"require": "./make-print-stuff.cjs",
"default": "./make-print-stuff.cjs",
"browser": "./make-print-stuff.mjs"
}
}
}
If users are in electron or other environment, they can choose to import "./make-print-stuff" to adapt.
For library provider: we can use “Solution Style” tsconfig.json Files to make ./feature-node.ts
include "types": ["node"] and ./feature-browser.ts
include "lib": ["dom"]
For library users: We can add a new tsconfig compiler options to choose import feature-node.mjs.d.ts
or feature-browser.mjs.d.ts
and so on.
Most helpful comment
For syntax, I'm pretty fine with just
and
because it mirrors our other shorthand
in which we just take the part of the declaration we do have (the name) and elide the body.