Ts-node: Top-level await support in REPL
Hey there,
Great work on ts-node! I just looked through the code a bit, and you all have clearly put a lot of thought and time into it. Thank you.
One request: I love using the REPL for quick exploration/experimentation/learning. And that often involves async steps, e.g. DB queries or API requests.
Before TypeScript, we used to use Streamline.js, and I loved that its REPL supported making async calls and awaiting their responses:
$ _node
_node> fs.stat('/dev/null', _)
{ dev: 1011986504, ... }
_node> r.get('https://httpbin.org/ip', _).body
'{\n "origin": "1.2.3.4"\n}\n'
_node> ...
I understand that TypeScript and ES6/7 as languages don't allow top-level await in modules, but would you be open to supporting that in the REPL?
The way the Streamline REPL does it is simply by wrapping the code in an async IIFE, and only calling the REPL eval callback once the async IIFE finishes:
https://github.com/Sage/streamlinejs/blob/v2.0.13/lib/repl.js#L23-L42
Thank you in advance for the consideration! Cheers.
aseemk
All 26 comments
Off then top of my head, I don't think it's possible to support without more hacking of the TypeScript compiler and/or into the future with ES6 modules. You need an async function for await to compile, but you can't import from inside a function (it must be at the top-level). Happy to keep the issue open though.
blakeembrey
on 3 Dec 2016
Thanks for the quick reply!
Could ts-node wrap in async IIFE only if the input had no imports? (And anything else problematic?)
It could still be really valuable to support top-level await for simple cases, even if it wasn't supported for more complex ones.
aseemk
on 3 Dec 2016
Maybe just .then() before printing?
> fetch('https://google.com').then(r => r.text())
... awaiting promise id: 2 (Ctrl-C to cancel)
'<HTML><HEAD><meta http-equiv="content-type" content="text/html;charset=utf-8">\n<TITLE>302 Moved</TITLE></HEAD><BODY>\n<H1>302 Moved</H1>\nThe document has moved\n<A HREF="http://www.google.co.nz/?gfe_rd=cr&ei=uK12WNbvMMTN8gfyir2ADw">here</A>.\n</BODY></HTML>'
Ideally do it more interactively, eg. show the prompt below the currently awaiting promises, print them as they arrive or something, but it's a start.
simonbuchan
on 11 Jan 2017
unional
on 22 Mar 2017
Temporary workaround which blocks the REPL while resolving:
npm install deasync --save-dev
# or
yarn add deasync --dev
const awaiter = promise => { const nothing = Symbol(); let ret = nothing; promise.then(response => { ret = response }); while(ret === nothing) { require('deasync').runLoopOnce(); } return ret; }
const value = awaiter(somePromise)
jessedvrs
on 28 Mar 2018
There is top level await in the node 10+ repl and it could theoretically be utilized running node --experimental-repl-await --eval 'require("./node_modules/ts-node/dist/bin.js")'. However, this is stopped by the TypeScript compiler complaining
> const a: string = await Promise.resolve("a")
[eval].ts(1,19): error TS1308: 'await' expression is only allowed within an async function.
which is totally right, since top level await is not (yet?) valid JavaScript but a hack for repl convenience.
I don't see a way to get the await keyword untouched through the TypeScript compiler.
webmaster128
on 20 Jul 2018
Updated version of @jessedvrs's workaround that works in a type safe fashion:
npm install deasync2 --save-dev
# or
yarn add deasync2 --dev
function wait<T>(promise: Promise<T>): T { return require('deasync2').await(promise); }
const value = wait(somePromise);
@webmaster128 If I understand it, that flag would not do anything for ts-node since it would only apply to the default REPL instance. We'd have to go ahead and implement something like https://github.com/nodejs/node/commit/eeab7bc0688256247c47099a90c67741e6637e42 ourselves here. It should be a bit easier to do for us, but we'd need to make sure we're handling all the same things.
blakeembrey
on 21 Jul 2018
@blakeembrey if you run node --experimental-repl-await --eval 'require("./node_modules/ts-node/dist/bin.js")', then you set process.binding('config').experimentalREPLAwait = true via the node command line (see https://github.com/nodejs/node/pull/19604). This runs our bin.ts, which then uses lib/repl.js I guess via import { start, Recoverable } from 'repl'. Thus I expect it to work automatically. I think the only one that stops this from working is the TypeScript compiler, that says top level await is invalid.
webmaster128
on 21 Jul 2018
@webmaster128 I think there's a misunderstanding. All the code to handle top-level await is within defaultEval, which we don't use because we aren't evaling as JavaScript directly. Therefore we need to replicate all the logic to handle this into ts-node - the flag won't do anything since it only applies to the node.js REPL and you aren't opening the node.js REPL. The TypeScript compiler fix is the trivial part, we just ignore that diagnostic in the REPL.
blakeembrey
on 21 Jul 2018
I think there's a misunderstanding. All the code to handle top-level await is within defaultEval, which we don't use because we aren't evaling as JavaScript directly.
You are probably right. I was a bit too optimistic here and did not follow the core carefully enough.
Would it be possible to implement the await keyword in the ts-node REPL using the wait<T>(promise: Promise<T>): T function from above?
webmaster128
on 21 Jul 2018
The TypeScript compiler fix is the trivial part, we just ignore that diagnostic in the REPL.
@blakeembrey could you elaborate on that part? I think I have a solution for running top level await in JavaScript utilizing the JavaScript AST (very similar to https://github.com/ef4/async-repl/blob/master/stubber.js and working in the ts-node REPL setup). However, I don't know how to teach typescript to pass the await untouched from TS code to JavaScript.
webmaster128
on 9 Aug 2018
There’s a feature in the README that allows you to ignore TypeScript diagnostic codes, it’d just be reusing that from the REPL.
blakeembrey
on 10 Aug 2018
Thanks @blakeembrey! To make top level await available available in JavaScript, you need to do two things:
- Ignore diagnostics
myTypeScriptService = register({
project: tsconfigPath,
ignoreDiagnostics: [
"1308", // TS1308: 'await' expression is only allowed within an async function.
],
});
- Set target to "es2017" or higher. Otherwise an awaiter is generated by TypeScript and
awaitis rewritten toyield
This works for simple cases like for the cases await 1 and await 1 + await 2.
In other scenarios, TypeScript treats await as a variable name, e.g. await (1):
TSError: ⨯ Unable to compile TypeScript:
[eval].ts(1,1): error TS2552: Cannot find name 'await'. Did you mean 'waits'?
webmaster128
on 14 Aug 2018
I don’t think we can rely on the target though, can we instead parse and wrap the file before giving to to TypeScript instead? That way, even if it generates the awaiter, the code output will be functional?
blakeembrey
on 15 Aug 2018
Top-level await seems to work at least to some extent with esm package after disabling both 1308 and 2304 diagnostics in source files. TS_NODE_IGNORE_DIAGNOSTICS=1308,2304 TS_NODE_COMPILER_OPTIONS='{"target":"es6"}' node -r esm -r ts-node/register topAwait.ts
Ignoring TS1308 is pretty simple and side-effect free since it seems to be related only to top-level await. Unfortunately TS2304 is required mainly due to error TS2304: Cannot find name 'await' and this suppresses all other 'cannot find name' errors.
It doesn't seem to work in command-line REPL though if started without a source file defined.
Pokute
on 23 Aug 2018
I don’t think we can rely on the target though, can we instead parse and wrap the file before giving to to TypeScript instead?
That would require this AST manipulation in TypeScript and not in JavaScript. I think the mayor challenge is how to expose local variables in TypeScript when rewriting:
let a = await 1;
a // 1
as
(async () {
let a = await 1;
})();
a // not available
what happens in the JavaScript version is that local declarations are converted to global properties (which only works in non-strict mode), e.g.
let a = await 1;
a // 1
becomes
(async () {
a = await 1;
})();
a // 1
@webmaster128 That's a very good point, I didn't consider how the previous JS approach got around this issue either. Why couldn't you do AST manipulation with TypeScript though? I'm not sure myself how mature that API is, but transforms do exist as a feature.
Another question, but how would const variables be possible here? I guess we can rely on TypeScript type checking to workaround. It looks like in Chrome (and maybe node.js, haven't tested yet), it's rewritten const so it's no longer a const when using await.
blakeembrey
on 27 Aug 2018
Why couldn't you do AST manipulation with TypeScript though?
Didn't try it.
It looks like in Chrome (and maybe node.js, haven't tested yet), it's rewritten const so it's no longer a const when using await.
Oh, interesting. I thought they had a more sophisticated approach allowing const. The method I use does not support const either on the JS side. But TypeScript takes care of that. But I don't think that converting top level let/const to var semantics in a REPL is a big deal.
Do you have an idea for valid TypeScript that exposes variables declared in an async function body to the global scope?
webmaster128
on 27 Aug 2018
Hello everyone, just stumbled on this one. Wondering if there are simple instructions on how to start a TS repl with ts-node and simply use async 100 without being hit with:
[eval].ts:1:1 - error TS1378: Top-level 'await' expressions are only allowed when the 'module' option is set to 'esnext' or 'system', and the 'target' option is set to 'es2017' or higher.
feliperyan
on 29 May 2020
Would love to know if there has been any updates to this one. Is it possible to do?
ilyakatz
on 1 Jul 2020
@feliperyan have you tried setting options they way the error message tells you to?
cspotcode
on 1 Jul 2020
Solutions mentioned here (ignore TS error + --experimental-repl-await) work with scripts but not in REPL mode.
guifel
on 5 Oct 2020
Does anyone know how node's REPL handles top-level await? Can we use their top-level await mechanism? Is it exposed as an API? Can we copy their source code?
Someone will need to propose how we can implement this.
cspotcode
on 5 Oct 2020
In case it helps anyone else, I've had good results in REPL mode with node --require ts-node/register/transpile-only --experimental-repl-await (using node 14)
dansebcar
on 6 Oct 2020
I was playing around with the repl today and managed to get it working. Looking at the issue about native ESM support with ts-node helped
Environment
Node.js Version: lts/fermium (v14.16.0)
Typescript Version: 4.2.2
Configuration
package.json (just the important bit to allow treating .js as ES6 modules):
{
"type": "module",
"main": "./src/index.js"
}
Typescript config (just the important bit):
{
"compilerOptions": {
"allowSyntheticDefaultImports": true,
"target": "es2017",
"module" "ESNext"
},
"include": ["src/**/*"]
}
Usage
If you are loading a typescript module that has a top level await:
node --experimental-repl-await --loader ts-node/esm ./src/clients.ts
node --experimental-repl-await --loader ts-node/esm
To import a module within the repl, use dynamic imports.
> const clients = await import('./src/clients.ts')
vamsiampolu
on 25 Feb 2021
Related issues
dakom
·
3Comments
sanex3339
·
4Comments
watzon
·
3Comments
joshua-tj
·
3Comments
JoseLion
·
3Comments
Most helpful comment
In case it helps anyone else, I've had good results in REPL mode with
node --require ts-node/register/transpile-only --experimental-repl-await(using node 14)