Typescript: Support multi-threaded compilation for --build

[vilic]

Search Terms

Project references, multi-thread, compilation, performance

Suggestion

We have migrated our main project to use project references. It simplifies the development workflow, but brings basically 0 noticeable performance improvement (for both local development and CI). There's even an illusion that now we get worse performance during development.

I guess the main reason behind this is lacking support for multi-thread compilation.

Use Cases

Improve compilation performance.

Examples

N/A

Checklist

My suggestion meets these guidelines:

• [x] This wouldn't be a breaking change in existing TypeScript/JavaScript code
• [x] This wouldn't change the runtime behavior of existing JavaScript code
• [x] This could be implemented without emitting different JS based on the types of the expressions
• [x] This isn't a runtime feature (e.g. library functionality, non-ECMAScript syntax with JavaScript output, etc.)
• [x] This feature would agree with the rest of TypeScript's Design Goals.

[RyanCavanaugh]

We'll be looking at multi-threaded stuff more when node's worker threads API leaves Experimental phase. It's certainly a ripe opportunity.

[DanielRosenwasser]

More keywords since I had trouble digging this up from #30900: multi-core multicore multithreading multithreaded multi-thread parallel concurrent spawn

[ericanderson]

Node 12 just came out and while worker threads are still experimental, they no longer require any special node command line args to enable them. I believe 11.7 no longer required --experimental-worker.

Given that node 12 will become LTS in 6 months, now is a pretty good time to test this out. I would also bet typescript would be the biggest user and could help drive the internals of this API.

[avilesj]

@DanielRosenwasser I think I found this issue faster thanks to you.

@ericanderson I agree with you. Node 12 is out now and presents a very good opportunity to try a feature like this. Modern consumer-grade computers have a lot of power to leverage!

[nomcopter]

My project's TS compilation slowed down by a fair margin after switching from parallel lerna builds to project references. Almost definitely from this lack of parallelization. Would love to reap the benefits of both!

[martinschleiss]

@RyanCavanaugh The worker threads API just left the experimental stage and were lifted to stable (see: https://nodejs.org/api/worker_threads.html). Finally!

Can we expect now that they will be integrated into the compiler in the near future?

[aaomidi]

This would be a great addition to typescript. Our builds are currently taking a good 5-6 minutes to complete.

[starpit]

as part of the parallelization push, it would be nice if tsc could handle inter-reference dependencies better. right now, to the best of my knowledge, i am required to specify a linearization of the dependence dag in my references list. otherwise, compilation errors (and rather catastrophic ones at that) result.

a parallelization-by-ply strategy of the dependence dag would solve both issues (parallelization, and dependence management); and, it would seem to me that some recognition of the dependence dag is necessary, in any case?

[Bnaya]

@starpit you need to specify only your direct dependencies,
as long your direct dependencies specify their dependencies correctly.
Regarding circularities support see:
https://github.com/microsoft/TypeScript/issues/33685

[starpit]

@starpit you need to specify only your direct dependencies,
as long your direct dependencies specify their dependencies correctly.
Regarding circularities support see:

33685

hi @Bnaya, thanks for the reply! what i meant was that, to the best of my knowledge, we have to place the references in a particular order. otherwise, tsc will fail in the cases where reference B comes after reference A in the references list, but reference A depends on reference B.

i was suggesting that any parallelization of the compiler will need to do a topological sort, anyway (i'm assuming that we will be doing a by-ply parallelization). if so, then we can also avoid the requirement that references be topologically sorted, by hand.

[Bnaya]

If reference B have reference A in his references,
The builder will pick it up when building up B, and build A before.
I've made many mistakes setting references correctly, and missing one reference can make your build sometimes fail and sometimes pass.
So i've created a small cli tool to help me set it up based on dependency graph made by yarn workspace
https://www.npmjs.com/package/typescript-monorepo-toolkit

[newtack]

Is this going to be part of 3.8?

Would love to get improved compilation speed sooner rather than later. Feature wise Typescript pretty much checks all the boxes (3.7 with optional chaining was last key feature missing), so from my side, focus on performance while ensuring it continues to be stable is most important.

[newtack]

Any update?

[yutamago]

I would also highly appreciate support for multi-threading.
Our working stations have anything between 6 and 16 cores and CPU utilization is way too low on all of them.

[timocov]

Just write my 2 cents about that.

_tl;dr: try to compile your project with https://github.com/timocov/parallel-typescript and see/share your results_ 😂

If we want to speed-up the build by parallelization, we need to share the state between workers/threads (parsed source files, configs, fs cache, etc). Afaik nodejs doesn't provide a way to share the same object between threads (please correct me if I'm wrong) so we can only post serialized object and de-serialize it in receiver (for that we need to have implemented something like this). If we don't share the state - we'll parse the same stuff again and again in every worker requires that stuff, which might affect the performance.

In other hand, if we can compile every file inside the project in parallel, but it requires either change the compiler API from synchronous to asynchronous (to be honest I don't believe that this will be done sometime 😂) or having a way to "stop the world and wait until other thread/process finished" (see @DanielRosenwasser's tweet about that).

If we don't share the state - we'll parse the same stuff again and again in every worker requires that stuff, which might affect the performance.

Because of that, it's possible (just possible) that multi-threaded compilation (in any kind: workers or processes) won't increase your build as much as you expect it (at least for some projects).

I have a small piece of code which run compilation in parallel for independent projects and assumes that incremental and tsbuildinfo options will help to speedup the compilation of the node sub-project (see example), but it doesn't (of course, if I don't make a big mistake). I tested that "tool" with compiling TypeScript compiler itself and lightweight-charts library. The total time of compilation full solution (root of all composite projects) without caches was worse than if I run it with just tsc -b:

• In case of lightweight-charts library the reason is that the tree of sub-projects' dependencies is just list (so no parallelization detected).
• In case of TypeScript compiler my hypothesis is a lot of projects depend on src/compiler sub-project, which has 600kb d.ts file, which should be parsed in every project which depend on it.

Some of my assumptions can be inaccurate (or even all of them), just my thoughts.

If you wish, you can try to compile your project with https://github.com/timocov/parallel-typescript (don't forget to run tsc -b --clean before) and see results (I'll be happy to help you with any kind of issues you might faced with it). If it increases your build somehow, I'll be happy to make it production ready as much as possible 🙂

[evandigby]

_tl;dr: try to compile your project with https://github.com/timocov/parallel-typescript and see/share your results_ 😂

@timocov Thank you for setting up that project. Just tested it out here on a TypeScript project with 253 packages on a Xeon W-2133 (6 cores @ 3.6ghz) with 32GB ram and got fairly good results (see below).

For a project like ours it looks like there'd be significant improvement (~30% faster) just implementing what you've done.

One thing worth noting is that it expectedly chews up my entire CPU. With the original 14 minute run my PC was still usable; however, on the 10 minute run it's completely unusable. It's clearly doing a bunch of redundant work, which is also expected from what you said. The workers version was slightly easier on the CPU, but only slightly.

To me this means our build machines would benefit more than our local machines. In our dev workflow it's relatively rare to have to do a full build of all of the projects.

Before:

> Measure-Command { tsc -b | Out-Host }

Days              : 0
Hours             : 0
Minutes           : 14
Seconds           : 9
Milliseconds      : 203
Ticks             : 8492037715
TotalDays         : 0.00982874735532407
TotalHours        : 0.235889936527778
TotalMinutes      : 14.1533961916667
TotalSeconds      : 849.2037715
TotalMilliseconds : 849203.7715

After (Processes):

> Measure-Command { node ..\..\parallel-typescript\index.js | Out-Host }

Days              : 0
Hours             : 0
Minutes           : 10
Seconds           : 9
Milliseconds      : 573
Ticks             : 6095731335
TotalDays         : 0.00705524460069444
TotalHours        : 0.169325870416667
TotalMinutes      : 10.159552225
TotalSeconds      : 609.5731335
TotalMilliseconds : 609573.1335

After (Workers):

> Measure-Command { node ..\..\parallel-typescript\index.js tsconfig.json node_modules/typescript/lib/tsc.js --workers | Out-Host }

Days              : 0
Hours             : 0
Minutes           : 9
Seconds           : 45
Milliseconds      : 854
Ticks             : 5858549663
TotalDays         : 0.00678072877662037
TotalHours        : 0.162737490638889
TotalMinutes      : 9.76424943833333
TotalSeconds      : 585.8549663
TotalMilliseconds : 585854.9663

[timocov]

significant improvement (~30% faster) just implementing what you've done.

Wow, that's really impressive. I didn't even think that I can find any project with improvement of speed here though 😂

One thing worth noting is that it expectedly chews up my entire CPU. With the original 14 minute run my PC was still usable; however, on the 10 minute run it's completely unusable

Yeah, I guess this is the cost of the parallelization.

To me this means our build machines would benefit more than our local machines. In our dev workflow it's relatively rare to have to do a full build of all of the projects.

Agree.

@evandigby is it possible to share "build flow" for your project (the tool prints it at the start before the compilation)? Anyway thank you for the feedback!

[evandigby]

@timocov No problem! Here's the build flow:

build flow: [13,4,1,11,3,6,3,2,40,23,30,22,8,22,22,12,11,6,1,6,3,1,1,1,1,1]

[KingOss]

@timocov this is not completely true

Afaik nodejs doesn't provide a way to share the same object between threads (please correct me if I'm wrong)

If it's true that you cannot share the same object (i.e. share the memory area of the object) between threads, you can always use messages to share the object, manipulate it and re-send back the edited one.

For example:

const { Worker, isMainThread, parentPort } = require('worker_threads');

var obj = { /* what you need */ };
if (isMainThread) {
  const worker = new Worker(__filename);
  worker.once('message', (message) => {
    // edit data here and use condition to avoid "infinite loop", e.g. wrap the object in a "message object" with the status or the "kind" of message
    worker.postMessage(message); // message should contain the edited object
  });
  // send first message to the worker process
  worker.postMessage(obj); // or, maybe is better to replace obj with { status: "status identifier", args: [ obj ] }
} else {
  parentPort.once('message', (message) => {
    // edit data here and use condition to avoid "infinite loop", e.g. wrap the object in a "message object" with the state or the "kind" of message
    parentPort.postMessage(message);
  });
}

It can be a little tricky, but it is possible, I've done it to achieve a (really heavily customized) parallel typescript build in my company. Actually we're building with 4 parallel threads, with around a 30-40% of speed up (but the customization slows down the build process of around the 20%).

_Note1: the code shown in the example has not been tested exactly as written and probably will require some fix to work properly. Sorry but I'm not authorized to copy-paste the code (and not even portions of it). For this example I've just edited the NodeJS documentation page to share my knowledge._

_Note2: I didn't fully understand why, but if you use multiple process build (using child_process) instead of workers, the build speeds up of around 10% more _ but the memory increase of around 25%. We're not using this ways because our the build process of our application uses around 4-6GB of memory (consider that the source code disk size is around 10GB and the "application" is composed of 12 "sub-application") and with more TSC processes we easily reach 8GB 😆_

Hope this can be useful! 😄

[timocov]

If it's true that you cannot share the same object (i.e. share the memory area of the object) between threads, you can always use messages to share the object, manipulate it and re-send back the edited one.

@KingOss Yes, I said that in case of sending/receiving the whole AST between workers to avoid unnecessary readings/parsings, e.g. you can't (or can?) send an object which has methods/functions in it.

It can be a little tricky, but it is possible, I've done it to achieve a (really heavily customized) parallel typescript build in my company.

Is it possible to open-source it?

[Bnaya]

@KingOss so i made this project for shared memory
Js objects, https://github.com/Bnaya/objectbuffer
I will be happy to give it a try with your tool