Ts-node: Compilation is unbelievably slow

[lukashavrlant]

Hi, for some reason we have performance problems when using ts-node in our mocha tests. It takes about 500 ms to compile an empty ts file. I added some console.logs to measure the time to run this very line: const output = service.getEmitOutput(fileName) and this line takes 500 ms to run even though the content of the file is an empty string (the code variable on the line 314 is empty string). Actually, all our tests files take too long to process, while the production files take a few milliseconds to compile. So any *.spec.ts takes about 500 ms to compile, a regular *.ts file takes about 20 ms. Do you have any idea what could be the root cause or how should I debug it more?

We use latest mocha, ts-node and typescript, but we tried some old versions too and the problem persists. The tsconfig.json:

{
  "compileOnSave": false,
  "compilerOptions": {
    "outDir": "dist",
    "rootDir": ".",
    "declaration": true,
    "target": "es2017",
    "lib": ["es2017"],
    "module": "commonjs",
    "moduleResolution": "node"
  },
  "exclude": [
    "node_modules",
    "dist"
  ]
}

[davidgruar]

I have also experienced this issue when running alsatian tests after upgrading to ts-node v7. I've found I can resolve the issue by specifying the --files CLI flag.

The problem seems to stem from the fact that the test framework requires each .test.ts file in turn, and each call to require appears to compile every module in that file's dependency tree from scratch. Specifying --files reverts to the old behaviour of loading all files on startup (and presumably caching them in memory).

@blakeembrey does this sound like a plausible diagnosis, and can you see any better solution?

[cspotcode]

Have you tried --transpile-only mode?

[mazswojejzony]

I've run into similar issue today. After ts-node upgrade from 7.0.1 to 8.0.1 my mocha test suite became very slow (32s vs 4s). Adding TS_NODE_TRANSPILE_ONLY=true resolved my issue. Thanks @cspotcode

[davidgruar]

--transpile-only does speed things up considerably for me. I don't want to lose the type checking though!

[lukashavrlant]

I tried all the proposed solutions. The files: true makes execution significantly faster! Turning off type checking also made things faster. I just had to turn off typeCheck and turn on transpileOnly (typeCheck: false, transpileOnly: true). This is good enough workaround, but it seems that the default behavior is still weird.

[blakeembrey]

That's intereseting, files: false is definitely meant to make things faster rather than slower. Does anyone have a large enough project that's running slow and want to either 1. investigate if the reason @davidgruar mentions appears true or 2. allow me temporary access to look into further myself?

Ideally files: false loads a single file and spiders out so it is possible that this is a slight issue with performance. It's also possible that the previous version was hitting the cache heavily, if you upgraded from v7 to v8 (v8 removed the cache because of type issues). We could also investigate adding caching back but it needs to be much smarter than the previous version to solve transitive type changes.

[garethflowers]

I've hit this issue too after updating from 7.0.1 to 8.0.1.

From what I can tell when files: false the tsconfig.json file gets changed in memory to remove the files and includes attributes.

https://github.com/TypeStrong/ts-node/blob/544851ee58af539908c682c547e05508e3dfb731/src/index.ts#L484

This seems to create an invalid config (when using TypeScript 3.2.4):

error TS18002: The 'files' list in config file 'tsconfig.json' is empty.

Removing the files and includes attributes altogether however seems to resolve the issue.

[blakeembrey]

If anyone can give me temporary access to a slow repo, let me know! I can quickly take a look into the issue and happy to sign any NDA, etc you need. If not, can I get a sense of peoples tsconfig.json that's slow - are you relying on files, include, exclude, rootDir or other to compile with TypeScript?

Let's try with reactions:

• ❤️ files
• 🚀 include
• 👀 exclude
• 😕 rootDir
• 🎉 Other

[garethflowers]

8.0.2 hasn't made any difference for me. The fact it's adding files: [] in to the tsconfig.json still kills performance.

I can't share our project unfortunately - but here's our tsconfig.json:

{
    "compileOnSave": true,
    "compilerOptions": {
        "emitDecoratorMetadata": true,
        "experimentalDecorators": true,
        "forceConsistentCasingInFileNames": true,
        "lib": [
            "es2017"
        ],
        "module": "commonjs",
        "moduleResolution": "node",
        "newLine": "LF",
        "noEmitOnError": true,
        "noImplicitReturns": true,
        "noUnusedLocals": true,
        "noUnusedParameters": false,
        "outDir": "dist",
        "removeComments": true,
        "rootDir": ".",
        "sourceMap": true,
        "strict": true,
        "strictFunctionTypes": false,
        "suppressImplicitAnyIndexErrors": true,
        "target": "es6"
    }
}

[blakeembrey]

@garethflowers Are you using mocha or something else that makes the explanation in https://github.com/TypeStrong/ts-node/issues/754#issuecomment-456788194 make sense?

Alternatively, can you try populating files = ['one.ts file here'] so it's valid, but still overridden, and let me know if that changes anything?

[davidgruar]

@blakeembrey I've made a minimal repro at https://github.com/davidgruar/tsnode-perf-repro - please check it out and have a play. Loading just three test files with fairly few dependencies takes more than a second, and the more imports you add, the longer it takes.

[blakeembrey]

It doesn't seem like a huge overhead when you time the entire program, instead of just requires:

There is a difference, however, so for testing it might make the most sense to enable --files whereas running an application it doesn't - there's only a single input and TypeScript pre-processes everything anyway. I'll look into it more though because the difference is clear and it would be nice to bring the dynamic files number down.

[blakeembrey]

Ok, perfect. It does look like TypeScript makes 4x the number of requests to the filesystem when files: false. Looking into whether there's some issue with how TypeScript is functioning here and how this could be improved.

[jeremy-coleman]

i've found several settings contributing to slow compile times, im pretty sure the issue is related to settings that will delete the default setting of ignore: node_modules. IE: allowJs. Others that may also (just from my memory of guess/check) are removeComments, emitDecoratorMetadata, and a few others. I've also noticed that if you glob without file ext like include: src//* instead of src//*.ts it can cause slowdowns , idk if its chokadir watching fontfiles that may be in there or something. using isolatedModules is the sure-fire way to speed things up though.

[blakeembrey]

@jeremy-coleman Sorry, I couldn't quite follow. Can you please let me know who you/what are responding to here?

Edit: I don't think any of those are related to the root cause of issues mentioned in this thread, that's all.

[lukashavrlant]

@blakeembrey Hi, I also created an example: https://github.com/lukashavrlant/ts-node-perf But I guess it is similar to the previous example. Basically compiling TS files to JS files and then running mocha tests using the generated JS files is faster then using ts-node directly. See readme for more details please. The more imports I use is test files, the slower it gets. For the sake of example I used zero imports in tests files.

[blakeembrey]

Unfortunately this definitely seems to be the case. I'm not 100% sure why in the language services this is so expensive. It appears that changing rootFiles results in the "project out of date" and it starts re-resolving all types and traversing node_modules. This is _very_ bad in the test loading case - I need to find a way to use the previously cached resolutions over trying to re-resolve everything when the root files change. cc @TypeStrong/typescript-team @weswigham in case I'm doing something wrong.

[weswigham]

@sheetalkamat would you know what would need to be done to allow API implementers to persist the resolution cache across builds?

[sheetalkamat]

When host doesn't have resolveModuleName it creates the cache per program. So whenever program changes the new resolution cache is created. (Note this is different from internal resolution cache that tsc and tsserver use that are made aware of which files have changed and hence which resolutions to invalidate) I don't think ModuleResolutionCache that compiler has, is equippd to handle the changes and hence its per program.
I think better option would be to use our WatchAPI instead as that is equipped to handle changes in file and hence invalidating partial resolutions.

[Orokon]

I'm also having issues with speed. I'm having a big Angular Application where I am running protractor tests with the jasmine framework. When I start the tests with protractor flag --troubleshoot i can see where it gets to the point where the files get processed. Usually that takes like +- 3 seconds and after that jasmine starts. Having version 8.0.0 and above its taking up to almost 2 Minutes until the files are processed and jasmine starts.

Sadly I don't know where I could try out the solution with file: false. Any help is appreciated.

[cezaryrk]

It's not a solution, but downgrade to 6.2.0 helped me currently with the test performace issues.

[blakeembrey]

For now, please try using the environment flag from https://github.com/TypeStrong/ts-node#cli-and-programmatic-options - TS_NODE_FILES=true. I'll attempt refactoring to the newer TypeScript watch API and see if it improves performance in the coming weeks.

[miao17game]

ts-node 7.0 -- 2.5s
ts-node 8.0 -- 5.2s

same environment.

[blakeembrey]

@miao17game By same environment, is this a test suite, a script or something else?

[chadbr]

Hi -- for what it's worth --

We have a rather large code base (no way to know really...)

It is on the order of 450 Mocha tests.

7.x was not "snappy" to startup - but it was tolerable.

I upgraded from 7.0.2 to 8.0.1 and the tests will actually never finish on Windows. They work fine on Linux (like 7.x).

After reading through this, I was trying to use the --files true flag. On a smaller codebase it made a big different (the tests ran).

On the code base with 450 tests... it simply never finished. The strange thing is if I set the environment variable TS_NODE_FILES -- set TS_NODE_FILES=true ; the tests start almost immediately.

If it helps:

mocha --recursive --require ts-node/register -R spec ./**/*.spec.ts --no-timeouts --exit

tsconfig.json

{
  "compilerOptions": {
    "target": "es6",
    "outDir": "dist",
    "rootDir": "src",
    "moduleResolution": "node",
    "module": "commonjs",
    "declaration": true,
    "importHelpers": true,
    "emitDecoratorMetadata": true,
    "experimentalDecorators": true,
    "sourceMap": true,
    "listFiles": false,
    "traceResolution": false,
    "pretty": true,

    "strictNullChecks": true,
    "forceConsistentCasingInFileNames": true,
    "noFallthroughCasesInSwitch": true,
    "noImplicitAny" : true,
    "noImplicitReturns": true,
    "noImplicitThis": true,
    "noUnusedLocals": false,
    "noUnusedParameters": false,

    "typeRoots": [
      "node_modules/@types"
    ],
    "lib": [
      "es2018",
      "es6",
      "dom",
      "dom.iterable"
    ]
  },
  "include": [
    "src/**/*.ts"
  ],
  "exclude": [
    "node_modules/**"
  ],
  "compileOnSave": false
}

package.json

    "ts-node": "^8.0.2",
    "typescript": "3.3.3"
    "tslib": "^1.9.3",

[kaiza]

We have just started running into this problem after upgrading our Angular app to 7.2.5 and at the same time we also upgraded ts-node to 8.0.2. Since the upgrade running the protractor e2e tests always took about 2 minutes before they would start running. After some investigation I found that it was taking a long time loading the spec files. I've had to downgrade ts-node back to 5.0.1 which fixes it but just wondering if anyone else has had the same problem and if they were able to solve it with the latest ts-node version.

[koliada]

I'm facing this issue when writing tests using mocha + proxyquire.
My solution so far is to mock as many dependencies with proxyquire as possible and use these three options for ts-node (they all seem to positively affect the start-up speed):

transpileOnly: true,
typeCheck: false,
files: false

[chadbr]

@kaiza set this environment variable -> set TS_NODE_FILES=true

It should make your tests 'go fast'

[jvanoostveen]

I was wondering if this is fixed in the new release (8.0.3).
We have a test set of nearly 8000 tests and with version 7 it only takes a few moments before it starts, but when I updated to version 8, it would take more than 30 seconds before the tests would start.

For reference: the codebase is about 70k lines of code, test codebase is something over 110k lines of code. With a total of almost 1800 files.

[OwenPattison]

@chadbr I'm having an issue with this too, I tried your suggested fix of adding the environment variable which unfortunately didn't work running ts-node 8.0.3

[DanielSchaffer]

@blakeembrey I'm hitting this problem with [email protected] in my monorepo project (https://github.com/just-dandi/dandi). I updated from 7.0.1 to 8.0.3 in this branch: https://github.com/just-dandi/dandi/tree/feature/9-pg-db-tests, and every time I run my test suite, it now takes around 30s. By comparison, on 7.0.1, it would run in ~6s once ts-node's cache was warmed.

I tried using TS_NODE_FILES=true, but it didn't seem to do anything.

If you'd like to try checking out that branch, just run npm run setup after cloning and then you should be able to run the tests with npm test.

[whxaxes]

I'm just using tsc --noEmit && TS_NODE_TRANSPILE_ONLY=true ts-node index.ts in my project, and it's 3x faster than ts-node index.ts ( there are 500+ ts files in my project. before: 96 seconds, after: 32 seconds ).

[zaggino]

also had to downgrade 8.0.3 to 7.0.1 to fix the performance issue, time to start tests was like 3 minutes vs few seconds difference, TS_NODE_FILES=true didn't help anything

[lukaselmer]

We had to downgrade too. This is what happens in our project 💥

7.0.1

8.0.3

Reproducing the issue

I also created a project to reproduce the issue: https://github.com/lukaselmer/reproduce-slow-ts-node

7.0.1

git clone https://github.com/lukaselmer/reproduce-slow-ts-node reproduce-slow-ts-node-v7
cd reproduce-slow-ts-node-v7
npm i
npm start
npm start

8.0.3

git clone https://github.com/lukaselmer/reproduce-slow-ts-node reproduce-slow-ts-node-v8
cd reproduce-slow-ts-node-v8
git checkout v8
npm i
npm start
npm start

With TS_NODE_FILES=true

With TS_NODE_TRANSPILE_ONLY=true

With TS_NODE_FILES=true TS_NODE_TRANSPILE_ONLY=true

It seems that even with the additional options, 8.0.3 is still 2-7 times slower than 7.0.1.

[jvanoostveen]

It doesn't seem fixed with version 8.1.0 😞

[kiliw]

I also have the performance issues while running tests with tape and ts-node 8.1.0. With the option TS_NODE_TRANSPILE_ONLY=true the tests run 6 times faster, whereas TS_NODE_FILES=true doesn't have any positive influence.

[blakeembrey]

@lukaselmer I tried your branch but it doesn't actually test any slowness except for using a filesystem cache. If you do --no-cache you end up with the same result. I'm not sure how many other people are requiring the same file contents over and over again - if they are we can do an in-memory cache for that but I suspect there's a different issue here not captured by your demo. If you do ts-node --no-cache in v7, you'll see similar performance numbers.

[blakeembrey]

@sheetalkamat FWIW, I tried the various options mentioned on that page but none of them fulfill the required use-case of ts-node. Watch mode 100% didn't work because I need to 1. get the source file which was always empty to be able to emit the specific file and 2. it took over the console to emit TypeScript debugging statements. I also don't think file watching is needed in ts-node, the issue is more that the TypeScript compiler will force the entire project to re-compute if the root files array changes (IIRC).

[blakeembrey]

@weswigham @sheetalkamat One big difference to perf is outputting all file names instead of the root file name (https://github.com/TypeStrong/ts-node/commit/6dad5ad34d0cea80ff9be75d9a77e07f19d9175a) - any ideas why that would make such a huge difference in performance? Aside from that, I went ahead and adding caching to all FS operations (https://github.com/TypeStrong/ts-node/commit/1e838f76a3b3590b3ee37acfcb9d846a4e0bd08d) - this makes a bigger difference for people using non-SSDs, but I found in the simple example above it makes 1400+ queries to "directory exists", 700+ to "file exists", 170+ to "read file", etc. Any clues on why this is necessary for the compiler? It seems like a lot of lookups - when cached it goes down to barely 40 fs operations in total.

[blakeembrey]

Please feel free to try 8.2.0, it has the two minor improvements above. It won't hit anywhere near as just re-using the cached emit multiple times yet since it shouldn't be overly common that people's files are exactly the same across their project.

[weswigham]

Any clues on why this is necessary for the compiler? It seems like a lot of lookups - when cached it goes down to barely 40 fs operations in total.

AFAIK we expect the CompilerHost provided implements appropriate caching (the builtin one based on sys does not) - we actually have a set of caching compiler hosts we use for tsc -b mode and the language service.

[sheetalkamat]

@blakeembrey as @weswigham suggested, we want CompilerHost to maintain all the caches. The improvements you are seeing, I think because of module resolution since that is what requires us to query directory/file at multiple locations.
For watch API, you want to provide watchFile and watchDirectory implementations which notify the watch about changes. Eg. https://github.com/sheetalkamat/ts-loader/blob/master/src/servicesHost.ts#L458 where it creates the watcher and webpack's known state of modified files/directories is notified to watch at https://github.com/sheetalkamat/ts-loader/blob/master/src/watch-run.ts#L51

[lukaselmer]

@lukaselmer I tried your branch but it doesn't actually test any slowness except for using a filesystem cache. If you do --no-cache you end up with the same result. I'm not sure how many other people are requiring the same file contents over and over again - if they are we can do an in-memory cache for that but I suspect there's a different issue here not captured by your demo. If you do ts-node --no-cache in v7, you'll see similar performance numbers.

I agree that with ts-node --no-cache it takes longer to load the files with version 7. I also tried it in our project.

• 7.0.1 normal (using the cache): Server startup: 1053.921ms
• 7.0.1 without the cache (ts-node --no-cache): Server startup: 24118.856ms
• 8.2.0 normal: Server startup: 13705.785ms
• 8.2.0 --no-cache command doesn't exist
• 8.2.0 with TS_NODE_TRANSPILE_ONLY=true: Server startup: 3117.804ms

The file system cache is already implemented in 8.2.0, right? Were there any other caches active in 7.0.1? 🤔

[blakeembrey]

@lukaselmer The cache 7.x is describing isn't to do with caching file system operations, but caching and checksumming the results from the TypeScript compiler. This is unstable since a change on a sub-sub-dependency could result in the current file failing, but using a cache this would go undetected (because the file didn't change). The right solution would be something like TypeScript's new incremental when it's exposed.

[lukaselmer]

@blakeembrey Thank you for the explanation.

[vort3xxx]

I also have the performance issues while running tests with tape and ts-node 8.1.0. With the option TS_NODE_TRANSPILE_ONLY=true the tests run 6 times faster, whereas TS_NODE_FILES=true doesn't have any positive influence.

@kiliw : That turns off type checking on the spec files. If you use any typescript types in your tests files, they're not going to be checked.

[kiliw]

@vort3xxx thanks for the info

[ccornici]

Same issue here, tsc takes 9 sec to compile an entire project, running only a subset of the tests takes 25 seconds to start them...
latest ts, latest mocha, latest ts-node, latest everything

For anyone looking to run their mocha tests very fast until this issue is fixed you can:

• run tsc --watchin a console window
• modify your test command to run the tests from the js output
• not register ts-node anymore to avoid the slowness

This way you can write your ts test, it gets compiled automatically to js, and when you run the test command it will execute it really quickly

[brunolm]

I'm having an weird issue where

console.log(new Date())
import * as express from 'express'
console.log(new Date())

is taking 2 seconds on my project, but on a clean one it takes just a few ms

I also noticed that TS_NODE_TRANSPILE_ONLY=true makes it slower and TS_NODE_FILES=true doesn't seem to do anything.

Any ideas?

    "ts-node": "^8.1.0",
    "typescript": "^3.4.5",

{
  "compilerOptions": {
    "module": "commonjs",
    "target": "es2016",
    "noImplicitAny": false,
    "sourceMap": true,
    "noUnusedParameters": true,
    "noUnusedLocals": true
  }
}

ts-node index.ts

Update:

ts-node@7 seems to be many times faster

[gruckion]

I have read over all the comments, the issue is still open and it's been close to 9 months. The latest version still has this problem and the only solution I can see here is to role back to 7.0.1

Are there any updates on this issue and when this will be resolved.

ts-node -r tsconfig-paths/register --files run-tests.ts | faucet

Using --files helps run the tests a bit faster for my colleague but for myself they do not complete even after 11 minutes whilst I was making tea.

[blakeembrey]

@gruckion If you want it to be improved, you'll need to submit a PR. Based on this discussion I'm unclear of how to improve the performance - the only possible configs are --files or --transpile-only. The only major gripe I can understand from comments is that people liked the file system cache a lot, which could be re-enabled but not as default because it causes subtle hard-to-debug issue. Using the new incremental compilation by TypeScript would stable and I'd love to see that implemented, I think that would resolve this thread!

[ronjouch]

Using the new incremental compilation by TypeScript would be stabler and I'd love to see that implemented, I think that would resolve this thread!

Passersby interested in implementing that, here's the announcement linking to PR & docs: TS 3.6 APIs to support --build and --incremental.

[connorjayfitzgerald]

I've been using 8.3.0 for a while. Tried upgrading to 8.4.1 today and startup time went from ~16s to ~75s.

[blakeembrey]

What about with 8.4.0? There’s only only code change which was performance based there, and it had to be reverted because it caused type errors that shouldn’t have been happening.

[connorjayfitzgerald]

Just tried 8.4.0 and that's starting up fine

[blakeembrey]

@connorjayfitzgerald Would you be willing to test https://github.com/TypeStrong/ts-node/tree/be/incremental-program and let me know what performance looks like for you?

[connorjayfitzgerald]

Sure. Still seeing the slow startup on that branch

[blakeembrey]

@connorjayfitzgerald This is why the issue is still open and needs help. I've tried a half dozen refactors (including the one above which rewrites the way TypeScript is used) and none see any measurable change in performance.

[DanielSchaffer]

is there a way to compare perf profiling from 8.4.0 and 8.4.1 to see what's causing the problem?

[blakeembrey]

@DanielSchaffer I'd love help with it, there's only one change (https://github.com/TypeStrong/ts-node/commit/87a6e2c5d73a43610bf21abde710a6c9ae973756) and it was reverted to fix https://github.com/TypeStrong/ts-node/issues/884. It's unclear how this negatively affected things, and it's also unclear why the changes in the incremental program aren't faster (they are in the test suite).

[ORESoftware]

maybe:

tsnode --optimistic

?

[LubomirIgonda1]

I can confirm a big performance difference between 8.4.1 and 8.4.0

my start script

cross-env NODE_ENV=development nodemon --inspect --watch src -r dotenv/config -r ts-node/register src/index.t

and tsconfig

{
"compilerOptions": {
    "esModuleInterop": true,
    "target": "es6",
    "module": "commonjs",
    "outDir": "./dist",
    "sourceMap": true,
    "noImplicitAny": true,
    "typeRoots" : [
        "node_modules/@types",
        "./src/types"
    ]
},
"include": [
    "./src/**/*"
],
"exclude": [
    "node_modules"
]
}

[christianrondeau]

Just a FYI, 8.4.0 takes approx. 5 seconds to launch our app, and 8.4.1 takes between 5 to 15 minutes to launch. We cannot share the project code to replicate, but hopefully this gives an idea of the slowdown it causes. Our project contains 245 .ts files.

[blakeembrey]

Are either of you able to test the incremental compiler API branch and let me know if it’s just as slow?

[christianrondeau]

@blakeembrey looks good when using this in our package.json:

  "ts-node": "https://github.com/TypeStrong/ts-node.git#be/incremental-program"

[blakeembrey]

That’s great feedback, thanks! If I can gather another piece of positive feedback in terms of performance on a large project, I’ll look into releasing this as v9 along with other breaking changes.

[connorjayfitzgerald]

@blakeembrey I've tried the incremental program branch and it launches our app in 10 or 11 seconds, whereas 8.4.0 takes around 16 seconds

[tstelzer]

I'm avoiding the issue by splitting compilation and the repl, as I still want type information:

package.json (cropped)

    "scripts": {
        "build:dev": "NODE_ENV=development tsc --watch",
        "start": "NODE_ENV=development nodemon -q node -r tsconfig-paths/register distribution/index.js --watch"
    },

and tsconfig.json (cropped)

        "pretty": true,
        "noEmitOnError": true,

then I run npm run build:dev in one terminal split, npm start in the other.

That's my workaround until this is fixed :man_shrugging:

[eliseumds]

@tstelzer I do something like that too.

[christianrondeau]

@blakeembrey I noticed the incremental branch has been deleted, but I don't see if it's part of a release? You said "v9", is this still your plan? I'm a little bit OC, so holding a package back is bugging my brain :D

[blakeembrey]

@christianrondeau It's currently on master but cannot support a use-case that used to be supported. I only discovered this after trying to test another bug in the backlog, so I haven't released it yet. Most likely I'll be looking to put it behind a flag as part of v8 for a bit.

[DanielSchaffer]

@blakeembrey I've got a fairly large public all-typescript monorepo (https://github.com/just-dandi/dandi) - is there anything I can do to help? I use ts-node to run my unit tests with mocha, and it's been pretty rough lately - it's just over 60 seconds from when I start mocha to when I start seeing results on my 2015 MacBook Pro.

For comparison, I can do a fresh build of the project (using tsc --build) in about 20 seconds. A rebuild takes about 7 seconds.

EDIT: Looks like this is limited to type checking - using --transpile-only is still pretty snappy

[DanielSchaffer]

I just tried 8.5.0 (by installing using the GH url), and it doesn't seem to be working for me at all now - my mocha ran for 5 mins without showing any results at all before I stopped it.

Also, I've been noticing that when I try to halt the process with CTRL+C, it doesn't always actually kill the underlying node process. I'm not sure if that's an issue with mocha, or something crazy going on with ts-node.

[foladipo]

When I upgraded our project's ts-node from 4.1.0 to 8.5.2, I also faced the above problem in that our app's startup time significantly slowed down. As advised above, using --transpile-only did significantly speed up the process, but I really didn't want to lose type-checking. So while I am waiting for a later version of ts-node to be released that will make our app build fast without having to use --transpile-only, I have downgraded to 7.0.1. That version gives me fast build times (because, for one, it caches parts of previous build results).

[leonardootoni]

When I upgraded our project's ts-node from 4.1.0 to 8.5.2, I also faced the above problem in that our app's startup time significantly slowed down. As advised above, using --transpile-only did significantly speed up the process, but I really didn't want to lose type-checking. So while I am waiting for a later version of ts-node to be released that will make our app build fast without having to use --transpile-only, I have downgraded to 7.0.1. That version gives me fast build times (because, for one, it caches parts of previous build results).

I did the same. I came from a pretty old version (3.3.0), and I was really surprised how bad the startup performance on 8.x is. Unfortunately, I had to downgrade as well.
@blakeembrey, any additional info about when a new release will improve these performance issues?

[foladipo]

That said, an ode to you @blakeembrey and the rest of the ts-node team. Your work on this free, open-source tool and your visibility/availability to its users is much appreciated.

[silveoj]

I tried all versions from 3 to 8. And some versions of nodemon 1-2.
Windows 7 and 10: it can take 10 seconds to build. But often it takes 4-5 hours to build (even rebuild after one single change).
The ways from this topic don't help yet.
~2100 ts files

But colleagues with Mac don't have this issue. Or did't notice it.
Thanks.

It's take us 10 sec - 1 hour if we use ts-node 3.* with added settings in tsconfig.json:

    "pretty": true,
    "noEmitOnError": true,

[cspotcode]

I created a very minimal reproduction here: https://github.com/TypeStrong/ts-node-repros/tree/754
It looks like require-ing each file with typechecking turned on eats up about 60ms, even if the file is basically empty.

Possibly related: discussion here about how getting diagnostics per file is apparently slow no matter what: https://github.com/typescript-eslint/typescript-eslint/issues/1335 But I don't know if I understand the details well enough.

[cspotcode]

I think I had a breakthrough. Compilation gets way faster when we implement getProjectVersion in our LanguageServiceHost. I think without that method implemented, the compiler has no choice but to routinely throw away a bunch of internal state. For example, for every required ts file, we make 2 calls back-to-back: one to get output JS, and the next to get diagnostics. Without implementing getProjectVersion, the compiler has to redo a bunch of work internally for both of those calls, which is crazy because nothing has changed.

For example, if you run the following assertion:

assert(languageService.getProgram() === languageService.getProgram())

... it fails in ts-node today, but it passes when I add an implementation of getProjectVersion().

TS source explaining getProjectVersion()

EDIT: Here's where this happens inside the compiler: https://github.com/microsoft/TypeScript/blob/master/src/services/services.ts#L1201-L1394
When project version is not the same or not implemented, a new Program and TypeChecker must be built.
Implemented and explained here: https://github.com/Microsoft/TypeScript/pull/3131/#issuecomment-103121030

Even when we don't provide a project version, the language service internally checks if the program is up-to-date using a more CPU-intense check, looping through all files, config values, etc, to see if anything changed. This check is failing, so there must be something else we're doing wrong.
https://github.com/microsoft/TypeScript/blob/master/src/compiler/program.ts#L555-L650

EDIT2: Found where it's failing. When any source files are omitted from getScriptFileNames the check fails because the internal HostCache doesn't have those files. Caused by the revert made between v8.4.0 and v8.4.1
https://github.com/TypeStrong/ts-node/issues/754#issuecomment-536097118
Maybe related: https://github.com/TypeStrong/ts-loader/issues/949

I filed a bug on Typescript to see if the Program invalidation is a bug on their end.
https://github.com/microsoft/TypeScript/issues/36748

[cspotcode]

I may have found a bug in the new incremental API implementation that causes it to drop back to the non-incremental API?

builderProgram is created via ts.createIncrementalProgram
https://github.com/TypeStrong/ts-node/blob/master/src/index.ts#L590-L605
...but here, it's reset with the legacy ts.createEmitAndSemanticDiagnosticsBuilderProgram
https://github.com/TypeStrong/ts-node/blob/master/src/index.ts#L625-L632

[blakeembrey]

@cspotcode That code is intentional and based on the typescript implementation (as far as I could gather at the time). You need to create a new program when the files expand in scope, but it’s built from the old program so shouldn’t be as slow. The incremental build compiler uses the same instance under the hood, but reads the TS build file first. Of course, I could have just implemented it incorrectly too.

[cspotcode]

@blakeembrey thanks, I'm learning a ton as I dig into the TS source, and I haven't looked at that bit yet.

I may have found a TS bug, (microsoft/TypeScript#36748) so fingers crossed they get back to me soon. Either way I think we can sidestep the bug my implementing getProjectVersion(). And I'm not sure it even affects the incremental build code-path. More to learn...

[jamemackson]

Were experiencing this issue as well and it's preventing our API to start within heroku's 60 second timeout for a dyno's start command. What's odd is that the same code starts in less then 5 seconds using ts-node-dev. This is using "ts-node": "^8.6.2", "ts-node-dev": "^1.0.0-pre.44"

Neither using --transpile-only or --files false helps in our case.

[brunolm]

@jamemackson we had the same issue. the solution was to downgrade to 7

[cspotcode]

@jamemackson @brunolm I'm almost positive the slowness should have been greatly reduced by transpile-only. Are you absolutely sure ts-node properly interpreted the --transpile-only flag and switched to using ts.transpileModule? It sounds like something went wrong with the flag, so it didn't switch to transpile-only mode, which is why it remained slow.

[jamemackson]

@jamemackson @brunolm I'm almost positive the slowness should have been greatly reduced by transpile-only. Are you absolutely sure ts-node properly interpreted the --transpile-only flag and switched to using ts.transpileModule? It sounds like something went wrong with the flag, so it _didn't_ switch to transpile-only mode, which is why it remained slow.

@cspotcode Do you know how can I check this? I just tried running it in heroku with the flag and compared the timing but don't have any visibility into what's going on internally.

[cspotcode]

@jamemackson Unfortunately I don't know of a great way to enable detailed logging from ts-node. We should probably add some sort of debug-logging flag that dumps out detailed diagnostics.

You could introduce an intentional type error that does not affect your project. transpile-only will allow it, but non-transpile-only will throw a type error. For example const a: number = 'this is a string';

EDIT: you could also wrap our require hook to log timing information, though this would not tell you whether or not ts-node was respecting the transpileOnly flag.

const orig = require.extensions['.ts'];
require.extensions['.ts'] = function(module, filename, ...rest) {
    const message = `.ts require hook for ${ filename }`;
    console.time(message);
    try {
        return orig.call(this, module, filename, ...rest);
    } finally {
            console.timeEnd(message);
    }
}

[DanielSchaffer]

@jamemackson How are you using ts-node? If you're using it via --require with a node executable, you may need to use the transpile-only registration entrypoint instead of the flag: --require ts-node/register/transpile-only

[jamemackson]

a flag to trigger detailed diagnostics would be great.

Forgive me since I know almost nothing about the internals of this but what i don't understand is that if ts-node-dev is using ts-node under the hood, why is it still fast when using the same ts-node version that is so slow when run on it's own 🤔

➜  api git:(test-performance) ✗ yarn start:dev
yarn run v1.22.0
$ ts-node-dev app/server.ts
Using ts-node version 8.6.2, typescript version 3.7.5

@DanielSchaffer we are running ts-node app/server.ts in our start script.

[cspotcode]

@jamemackson ts-node app/server.ts will not do transpile-only mode. Is there something else you're doing to put it in transpile-only mode? Basically I'm trying to understand exactly what's happening in your situation, but I think there's more information needed to suss that out.

Looks like ts-node-dev uses some sort of caching layer. I don't know anything about it: not what it's doing, what it's storing, or when it's invalidated. So I don't know how it affects performance, if at all.

[jamemackson]

I'm sorry, that's our normal start script, I did add the --transpile-only argument to that when I was testing this.

[cspotcode]

@jamemackson I spoke too soon about debugging. There's a TS_NODE_DEBUG env var you can set to get additional output from ts-node. It's not documented, and it doesn't log much so we'd probably need to add more log statements.

[jamemackson]

Thanks @cspotcode. I emailed you some logs. I hope it helps! Thanks for all your help!

[cspotcode]

963 implements getProjectVersion, which should workaround microsoft/TypeScript#36748, meaning the compiler needs to typecheck 3x less than before. This will only affect you if you have not enabled transpileOnly / --transpile-only / TS_NODE_TRANSPILE_ONLY nor compilerHost / --compiler-host / TS_NODE_COMPILER_HOST.

If you're really eager or curious, you can test it by putting this in your package.json deps: "ts-node": "https://github.com/TypeStrong/ts-node.git#ab/add-getprojectversion",

Setting TS_NODE_DEBUG=true will emit extra debug statements related to when and how often the compiler needs to rebuild the typechecker.

[cspotcode]

I'm digging into the codepath that uses the incremental compiler API. It was really slow for my reproduction (link), like a second per require(), because for every single change to the Program, it was re-parsing all SourceFile instances. Our non-incremental codepath uses a LanguageService, which uses a DocumentRegistry to cache these.

For the incremental compiler, we have to pass a CompilerHost, which has a getSourceFile method. This was always returning a new SourceFile instance. I don't know if this is intended behavior or if there's a way to cache SourceFile instances. (or if I'm doing something wrong in my experiments)

[pscanf]

I tested the branch and it does help quite a bit!

My results:

• with ts-node v8.6.2: times 32.56 / 32.67 / 31.71 => avg 32.31
• with the fix: times 23.84 / 23.28 / 23.70 => avg 23.61

There's an even bigger improvement using the --watch mocha option, which re-runs tests on file changes:

• with ts-node v8.6.2: time from file save to test beginning to re-run ~15s
• with the fix: time from file save to test beginning to re-run ~2s (with the first re-run taking a bit more, ~3.5s)

Technical details

OS: macOS v10.14.6

Node version: v13.8.0

Test command: cross-env NODE_PATH=src NODE_ENV=test TS_NODE_FILES=true mocha --exit -r ts-node/register --extension ts "test/*/.ts"

Amazing work, thanks!

[cspotcode]

Dumping more links:
I asked in Discord about the shouldCreateNewSourceFile arg of CompilerHost::getSourceFile.
orta linked me to https://github.com/microsoft/TypeScript-wiki/pull/225
related discussion: https://github.com/microsoft/TypeScript/issues/31849

Only place the TS codebase uses that arg: https://github.com/microsoft/TypeScript/blob/master/src/compiler/watchPublic.ts#L467

[diegolaciar]

I use nodemon and tsnode

I updated to "ts-node": "^8.6.2", and on each restart takes a lot to compile!

With TS_NODE_FILES=true did not show any kind of improvement.

I will try downgrade to previous version "ts-node": "^7.0.1" or 8.4.0 as @christianrondeau mentioned from 8.4.1 version breaks performance.

[vort3xxx]

My closed source project was recently updated from a custom webpack config to using angular-cli as its task runner, and one striking thing that happened was that its typescript compilation time went way down, like from 12 minutes to 3 minutes.

The differences were really subtle, and one thing that winked at me was the way that files were sent to tsc. In my old version of the project, I was using file globbing masks to load files, src/**/*.spec.ts, for example. In the newer angular-cli based code, the tool generates a webpack.config on demand, and within it, there is an explicit array of filename strings that get passed as arguments to the compiler. Each line item is a fully qualified file name.

It seems like projects with a lot of files started type checking much slower shortly before typescript 3.x was released and the angular team may have found a way to buy back some performance with that explicit file list trick.

Perhaps it would be worth this team benchmarking a list of files vs a glob of files to see if this project is also affected by the behavior.

[cspotcode]

Please try ts-node v8.7.0. It's been hanging out on the next dist-tag but I just promoted it to latest.

https://github.com/TypeStrong/ts-node/releases/tag/v8.7.0

The LanguageServiceHost::getProjectVersion implementation should help some projects quite a bit.

[brunolm]

❌ [email protected]

• 42s
• 41s
• 40s

❌ [email protected]

• 48s
• 47s
• 47s

✅ [email protected]

• 7s
• 3s
• 4s

[diegolaciar]

Hi @brunolm ,

How do you do to measure execution time?

8.4.0 works fine for me , I do not notice difference with 7.0.1 ,
8.6.2 is veeeery slow. I didn't tried the v8.7.0 yet

[brunolm]

I have one big project, that unfortunately I can't share. The entry point is "start"

"start": "ts-node src/index.ts"

At the top of this file (after many imports) I have a console.log(new Date())

Then I run

date; npm start

And I get the diff from there.

---

✅ [email protected]

• 10s
• 9s
• 9s

[blakeembrey]

@brunolm Your update on 8.4 inspired me to revisit the change in that version. Can you try https://github.com/TypeStrong/ts-node/pull/985 at all?

[brunolm]

@blakeembrey

✅ https://github.com/TypeStrong/ts-node/pull/985 (npm i -S TypeStrong/ts-node#be/improve-perf-again)

• 10s
• 9s
• 9s
• 9s
• 9s
• 10s

Looks like you nailed it!

❤ 😄 🎉

[blakeembrey]

Released as 8.8.0, please let me know if anyone sees a regression in other parts of the application (or errors with TypeScript as happened in 8.4.0).

[foladipo]

Big kudos to you @blakeembrey for sticking with this problem this far! I have not tried out 8.8.0 yet, but the reactions that people have made about it so far assure me that you have indeed solved the problem.

Ẹ ṣeun pupọ! Thank you very much!

[laughinghan]

(Update: The performance gap with v7 is much, much smaller with a cold disk cache. I think I will upgrade to v8.8.1 after all. For posterity, I'm on a macOS 10.15.3, MacBook Pro 15in Late 2013 with SSD.)

Hi, if you'd prefer I open another issue let me know, but for my project (which is open source, so hurray for being able to repro!), neither v8.8.1 nor reverting to v8.4.0 are remotely as big of performance improvement for as reverting to v7.0.1.

However, I think my case is different from the others, because my project is very small, and only takes a couple seconds to compile regardless:

• ~2.0s @ v8.7.0 - v8.8.1 (improvement over v8.6, nice work!)
  
  
  
  • ~1.2s with TS_NODE_TRANSPILE_ONLY=true
  
  
• ~2.3s @ v8.4.1 - v8.6.2
  
  
  
  • ~1.2s with TS_NODE_TRANSPILE_ONLY=true
  
  
• ~2.2s @ v8.4.0
  
  
  
  • ~1.2s with TS_NODE_TRANSPILE_ONLY=true
  
  
• ~1.9s @ v7.0.1 (first time after editing either of my .ts files)
  
  
  
  • ~1.1s with TS_NODE_TRANSPILE_ONLY=true
  
  
• ~0.8s @ v7.0.1 (subsequent runs, which is not that useful)
  
  
  
  • ~0.8s with TS_NODE_TRANSPILE_ONLY=true
  
  

Is there any reason I shouldn't just stick with v7 for now?

[sylc]

On the project i'm working on I have noted an issue with ts-node 8.8.1 and typescript 3.8.3

We were using ts-node v7.0.1 and typescript 3.5.3 and it was taking 22 mins on a clone repo for ts-node to start.
Thanks to v8.8.1, still with types checking and typescript 3.5.3, it takes now 38s to start.

However, now upgrading typescript to v3.8.3, it is taking 7 mins to start.
Downgrading to typescript v3.7.5, it is working again at 38s.

Using TS_NODE_DEBUG on the v3.8.3 version i can see the same symptoms as previously. the following lines are really slow to process.

ts-node invariant: Is service.getProject() identical before and after getting emit output and diagnostics? (should always be true)  true
ts-node module._compile ….

[cspotcode]

To clarify, when you say those lines are slow to process, what do you mean? Do you mean that whatever happens after those lines appear is taking a long time, or whatever happens immediately before those lines appear is taking a long time?

It might help if our debug logging included timestamps.

[sylc]

I mean that there are 100s of those lines and each of them are really slow to process, compare to with previous version of typescript.
See below extract of logging with timestamp

[ts-node 2020-04-01T12:09:10.565Z] invariant: Is service.getProject() identical before and after getting emit output and diagnostics? (should always be true)  true
[ts-node 2020-04-01T12:09:10.568Z] module._compile fileA.ts
[ts-node 2020-04-01T12:09:11.736Z] compiler rebuilt Program instance when getting output for fileA.ts
[ts-node 2020-04-01T12:09:11.786Z] invariant: Is service.getProject() identical before and after getting emit output and diagnostics? (should always be true)  true
[ts-node 2020-04-01T12:09:11.788Z] module._compile fileB.ts
[ts-node 2020-04-01T12:09:13.225Z] compiler rebuilt Program instance when getting output for fileB.ts
[ts-node 2020-04-01T12:09:13.273Z] invariant: Is service.getProject() identical before and after getting emit output and diagnostics? (should always be true)  true
[ts-node 2020-04-01T12:09:13.275Z] module._compile fileC.ts
[ts-node 2020-04-01T12:09:14.595Z] compiler rebuilt Program instance when getting output for fileC.ts
[ts-node 2020-04-01T12:09:14.841Z] invariant: Is service.getProject() identical before and after getting emit output and diagnostics? (should always be true)  true
[ts-node 2020-04-01T12:09:14.845Z] module._compile fileD.ts
[ts-node 2020-04-01T12:09:16.255Z] compiler rebuilt Program instance when getting output for fileD.ts
[ts-node 2020-04-01T12:09:16.268Z] invariant: Is service.getProject() identical before and after getting emit output and diagnostics? (should always be true)  true
[ts-node 2020-04-01T12:09:16.270Z] module._compile fileE.ts
[ts-node 2020-04-01T12:09:17.718Z] compiler rebuilt Program instance when getting output for fileE.ts
[ts-node 2020-04-01T12:09:17.828Z] invariant: Is service.getProject() identical before and after getting emit output and diagnostics? (should always be true)  true
[ts-node 2020-04-01T12:09:17.892Z] module._compile fileF.ts
[ts-node 2020-04-01T12:09:19.138Z] compiler rebuilt Program instance when getting output for fileF.ts
[ts-node 2020-04-01T12:09:19.149Z] invariant: Is service.getProject() identical before and after getting emit output and diagnostics? (should always be true)  true
[ts-node 2020-04-01T12:09:19.152Z] module._compile FileG.ts

let me know if you need anything else. Unfortunatly I cannot share the project.

[cspotcode]

@sylc I don't know exactly why it's slower compared to the previous version of typescript. But I can offer some insight as to how the compiler works.

The Program is immutable. Whenever a file is added to compilation, a new Program needs to be created to include the new file. The old ASTs can be reused so that the compiler doesn't need to re-parse them.

However, the typechecker must be thrown away and a new one constructed. This means every time a file is added to the program, typechecking must be redone from scratch. (at least as I understand it) This is necessary because any file might include global type declarations or declaration merging that affects any other file of the program.

If you haven't specified the "types" array in your tsconfig, it's possible that the compiler has to load many, many @types declarations in addition to your codebase. This can add up to a ton of typechecking.

Any time it says "rebuilt Program instance" this is happening.

We can avoid this by not typechecking via --transpile-only mode. We can run tsc --noEmit separately to handle typechecking instead of ts-node.

Another option is using ts-node's --files flag to eagerly load all your code at the beginning, so that the Program instance and typechecker don't need to be rebuilt. If more files are loaded at the beginning, then they won't be new to the Program when they're require()d, and typechecking won't need to be recomputed. If you manage to get this working, most of the "rebuilt Program instance" lines should go away.

[sylc]

@cspotcode thanks for the insight.
I did some digging. It is working fine on linux. The issue is only on windows.

I believe the difference in behavior between the 2 typescripts version come from this.
https://github.com/microsoft/TypeScript/pull/36011
especially https://github.com/microsoft/TypeScript/blob/0b38a9a2b03d3c651822bc2a20d381545384f0f5/src/compiler/program.ts#L566

When i track it down it is rebuilding because when typescript does !arrayIsEqualTo(program.getRootFileNames(), rootFileNames)
it found that C:\project\fileA.ts is not equal to C:/project/fileA.ts

Any suggestion for a fix?

[blakeembrey]

@sylc Is the forward slashes coming from TypeScript? One thing I've had to do a bunch in ts-node is wrap anywhere we pass filenames in normalizeSlashes. If you're willing to fork this repo and find the missing culprit, that'd be really helpful (as I'm not using Windows)!

[blakeembrey]

Here's an example: https://github.com/TypeStrong/ts-node/blob/92cf9613828c2ba2fe37b7a0aa7895e122c16185/src/index.ts#L606. It'd probably be worth figuring out if other cache hits like https://github.com/TypeStrong/ts-node/blob/92cf9613828c2ba2fe37b7a0aa7895e122c16185/src/index.ts#L585 are even working or if it's always falling back to disk.

[cspotcode]

@sylc Thanks for investigating!

FWIW, I normally use WSL2 (Ubuntu) but I have access to Windows, so if you create any reproductions I will be able to run them locally.

[cspotcode]

We implement getProgramVersion to tell the typescript compiler when we know nothing has changed. The only time typescript should be doing !arrayIsEqualTo(program.getRootFileNames(), rootFileNames) is when we know the Program must be rebuilt.

I wonder if the filename inconsistency means it has to re-parse all those source files every time it rebuilds the Program, rather than reusing the old ASTs? We added getProgramVersion for exactly that reason: it was re-parsing all source files every time, which is what made it so slow. Maybe that's still happening on Windows.

[sylc]

@cspotcode I have made a minimal reproduction project. see README at https://github.com/sylc/ts-node-debug

[cspotcode]

@sylc great. On Windows, does the compiler expect all filenames to use backslashes or forward slashes?

Does the reproduction prove that the bug is causing the slowdown? Does it get noticeably faster when #997 is merged?

I think to reproduce the slowdown, we can generate a couple hundred large, meaningless Typescript files. These files will simulate a large project, with lots of code to parse (for example, maybe each contains hundreds of lines of while(false) {/* do nothing */}). And we'll have a single entry-point with hundreds of import statements. import {} from './1'; import {} from './2'; That way, we can get some meaningful logs with timing information.

[DanielSchaffer]

@cspotcode if you want a large real life typescript project (with tests!), you can use this one that I maintain: https://github.com/just-dandi/dandi

[sylc]

@cspotcode

Does the reproduction prove that the bug is causing the slowdown? Does it get noticeably faster when #997 is merged?

I tried my branch on my work project and find that the speed was back to normal, i.e the same as tsc 3.7.5.
So, with tsc 3.8.3, 6 mins without my changes versus 50s with my changes

On Windows, does the compiler expect all filenames to use backslashes or forward slashes?

It can handle both as it is normalising the path as well. see some related tests I think the index of the cache could potentially have backward slashes, as long as it is always consistent. But it make sense to make everything forward slashes.

@DanielRosenwasser I had a quick try at your repo (clone, then npm i then nm run test or npm run build) but i'm getting some errors (builder not defined). feel free to try my branch and report back on the times.

[brunolm]

Sorry to jump in here this might be just my opinion, but I think the original issue "Compilation is unbelievably slow" has being fixed. It at least for the most part went back to normal.

I was just wondering if this issue should be closed and a new one opened to go about this other issue you're talking about now. If that makes sense.

[greenreign]

Sorry to jump in here this might be just my opinion, but I think the original issue "Compilation is unbelievably slow" has being fixed. It at least for the most part went back to normal.

@brunolm Sorry, I'm trying to follow this thread but I didn't see this. What was the fix? I'd love to try it on my project.

[DanielRosenwasser]

@sylc I think you meant to tag @DanielSchaffer

[greenreign]

Ok, I think I found it.

Thanks to v8.8.1, still with types checking and typescript 3.5.3, it takes now 38s to start.

Upgrading from 8.6.2 to 8.8.2 does seem to fix my slowness. Or at least most of it. I had a single module that does some scanning. It took ~130ms with tsc/node and took ~15s with ts-node 8.6.2. It's now taking ~450ms with 8.8.2.

I'm using windows and typescript 3.7.5

NOTE this only seem to improve if I have the --files option.

[brunolm]

A little summary of this thread so far:

https://github.com/TypeStrong/ts-node/issues/754#issuecomment-601893941 compilation with ts-node 8.7.0 and 8.6.2 was taking +40~50s compared to 7.0.1

https://github.com/TypeStrong/ts-node/issues/754#issuecomment-601902386 I was asked to test with 8.4.0 and it was only +3~6s slower than 7.0.1

https://github.com/TypeStrong/ts-node/issues/754#issuecomment-602048372 I tested 8.8.0 from the PR branch and the performance was equivalent to 8.4.0

https://github.com/TypeStrong/ts-node/issues/754#issuecomment-602119414 8.8.0 was deployed fixing the unbelievably slow issue (+40~50s)

---

I haven't experienced any unbelievably slow issues after that.

[cspotcode]

Whether or not the original slowness issue has been solved, I'm happy to leave this ticket open a bit longer, because the discussion is still useful. Ultimately the goal with these tickets is to help us make ts-node better, and it's still serving that purpose for me.

Where individual issues are being identified, we're tracking and fixing them as separate tickets. Additionally, the recent change in 8.8.0 may need to be reverted since it triggers the regression test from #884 So it's possible we're not out of the woods yet.

See also: #996, which links to the other relevant tickets.

Also, specifying whether or not you're on Windows is very helpful! Since it seems we may have a windows-only performance problem related to / vs \ paths.

[DanielSchaffer]

@sylc sorry, I just realized I don't have any setup instructions in the README. They should be in the README on develop now.

First, I'm primary set up using Yarn. You might be able to make it work with npm, but you'd need to replace a couple usages in package.json (e.g. builder yarn install -> builder npm install) and the setup script (e.g. safe_run yarn -> safe_run npm i, yarn link -> npm link, yarn install-packages -> npm run install-packages). Not sure off the top of my head if there are any others.

To make sure all the subprojects get set up and installed correctly, run yarn setup (or npm run setup if you choose to try that route). After that, yarn build and yarn test (npm run build / npm test) should do their thing as expected.

[brunolm]

@cspotcode cool! I was just wondering if it would make sense to track in other issues, but if this one is still relevant then it makes sense to keep it open.

In my case I was having issues on my machine (Windows 10 Pro x64), but also on Heroku, the app was taking over 60s (their default timeout) to startup. And it was also slower on Mac (I don't have access to one right now to test it and get the numbers tho).

[DanielSchaffer]

How can I test cached vs uncached runs in v8.0+? Looks like older versions had a --cache-directory flag (or cacheDirectory configuration option) that controlled where the cache went, but those don't seem to have an effect in the 8.x line (no cache files are created). I'm also not seeing any difference setting the TS_NODE_CACHE env variable.

[cspotcode]

@DanielSchaffer caching was removed once upon a time. There are plans to re-add it, and issues tracking that work.

[martinblostein]

I use node-demon with ts-node for automatic restart of the server on code edit. I still see a consistent 10x slow down for restart, between versions v7.0.1 and v8.10.1 of ts-node. I am using Linux Mint.

I will try and create a minimal reproducible example, but I just wanted to note that the speed regression is still very clear, at least on my machine.

[brunolm]

@martinblostein could you share your environment settings? It might be useful for them.
OS, Node version, npm version, etc...

[cspotcode]

@martinblostein I suspect it's faster because you have caching enabled in 7. This was removed in 8.
See #701 for the reasons why.

To effectively compare the performance, can you try again with environment variable TS_NODE_CACHE set to false.

[martinblostein]

@cspotcode Ah, yes you are correct. If I disable caching then version v7 takes as long as v8.

I think we will stick with v7 keeping that issue in mind though. Thanks for bringing it to my attention!

[david-wb]

I see this thread hasn't been active for several months, but I'm experiencing extremely slow unit tests when using mocha + ts-node. Did anyone figure out the root cause?

I tried many of the suggestions above, including the transpile-only and files options, without luck. I also observed that running one test by itself is fast (using mocha test explorer in vscode), but running all tests together is extremely slow (~2s per test). So it seems that mocha + ts-node might be re-compiling all of the test files for each and every test. Is there a way to avoid that?

[cspotcode]

@david-wb Mocha and ts-node do not recompile files repeatedly by default; there would have to be something else in your tests causing that to happen. Without an example we can't know, so if this is affecting you, I recommend sharing more details.

Another possibility, maybe you didn't successfully enable transpile-only mode. If you thought you did, but actually did not, then that would explain why it didn't get faster. Maybe the flag was passed wrong, etc. Without an example I can't say.

970 and #1092 should speed up incremental compilation by making us smarter about changing rootFiles.

There is a ready-to-use caching solution in #908 that you can try today. You can also help us to re-add caching as a native ts-node feature.

If you want to help out, let us know and we can provide guidance in those tickets.

[sandorvasas]

Using ts-node 7.0.1, nowadays it takes 15+ minutes for our API to boot! --transpile-only helps with the speed, but type-checking is a must for us, and so I can't use --transpile-only.

[cspotcode]

@sandorvasas
Why use ts-node v7 instead of v9? Is something preventing you from using v9?

What are you using to do typechecking today? I assume you're using tsc --noEmit. Is it working for you?

[cspotcode]

@sandorvasas have you been able to try version 9?

[mewtlu]

@cspotcode should we expect improved performance in version 9? I don't see anything regarding anything of the sort in the changelogs, although having just ran it on one of my projects with which I have performance issues when using older versions, and compilation certainly seems to have sped up, even now running faster than ts-node-dev, impressively!

[cspotcode]

@mewtlu that's great to hear. We fixed a few issues which I think were causing TS to unnecessarily repeat a bunch of work, so you may be benefitting from those changes.

For example, we're smarter about tracking which files are already loaded by the compiler, so we can avoid modifying the program to force their inclusion. This was causing a lot of duplicated work inside the compiler.

[sandorvasas]

I just tried 9, and altough it feels somewhat faster on the initial compilation, it recompiles at almost the same speed as the initial on file changes, which overall makes it much less useable than v7 (need to wait 4 mins to recompile after changing a single character in code). I switched back to v7.

[acro5piano]

I'm thinking to switch plain tsc -w and nodemon combination with "incremental": true.

I love to use ts-node, but it tasks 8 seconds to start our API, whereas 3 seconds for plain tsc.

[Alexandredc]

@acro5piano could you explain how you do that please 🙏

[khaledosman]

@acro5piano could you explain how you do that please 🙏

@Alexandredc instead of having your nodemon config watch your .ts files and execute ts-node on the entry ts file on changes, You can instead run tsc --watch (optionally with "incremental": true in your tsconfig.json) when the .ts files change to generate the js files in the "outputDir" folder, then run node / nodemon against the generated entry .js file

[acro5piano]

@khaledosman @Alexandredc

My current solution is tsc + nodemon, but I'm still doing the configuration for Ava.
The downside is the complex configuration. ts-node and tsconfig-paths is a simple configuration, works nicely for fs.readFile and ts path mappings and Ava. But tsc + nodemon is 2x speeds up my project.

The basics are:

# Directory structure

repo/
   |- tsconfig.json
   |- package.json
   |- src/
   |    `- index.ts
   `- build/
        `- index.js

// tsconfig.json

{
  "compilerOptions": {
    "outDir": "./build",      // <-- important
    "noEmit": false,     // <-- important
    "allowJs": true,
    "target": "es2019",
    "lib": ["dom", "esnext"],
    "module": "commonjs",
    "declaration": false,
    "declarationMap": false,
    "sourceMap": false,
    "removeComments": false,
    "incremental": true,
    "isolatedModules": true,
    "strict": true,
    "moduleResolution": "node",
    "allowSyntheticDefaultImports": true,
    "esModuleInterop": true,
    "experimentalDecorators": true,
    "emitDecoratorMetadata": true,
    "skipLibCheck": true,
    "forceConsistentCasingInFileNames": true
  }
}

// package.json

  "scripts": {
    "dev": "concurrently yarn:dev:*",
    "dev:nodemon": "nodemon build/index.js",
    "dev:tsc": "tsc -w"
  },

And the last thing is to add deps:！

yarn add -D concurrently nodemon typescript

[azu]

I've met same issue on developing web app/API server using express + ts-node-dev(ts-node).
So, I've created express-lazy-router for reducing this issue.
This is inspired by webpack's lazyCompilation.

express-lazy-router help us to reduce the compilation time at start time by compiled when needed.
📝 It is for webapp using express, it is not for testing

[kfrajtak]

Can you use the power of esbuild? https://esbuild.github.io/

[acro5piano]

@kfrajtak
I've tried esbuild and it is unbelievably fast.
I moved to it. Compile + run is much faster than ts-node. Thanks.

Edit: I'm using esbuild --watch and nodemon build/out.js now

Edit 2: I've tried to use other project using decorator, but its Interpolation is still different from original tsc. so I'm sticking tsc --watch in that project.

[cspotcode]

See also: #1160 which allows opting into the swc compiler via ts-node.

[acro5piano]

Could esbuild-register be an alternative solution?

https://github.com/egoist/esbuild-register

[cspotcode]

I investigated esbuild when implementing #1160 but there were performance limitations in per-file compilation. #1160 gives the same performance benefits, and people are already successfully using it, for example https://github.com/TypeStrong/ts-node/discussions/1276#discussioncomment-590910 I recommend checking it out and giving feedback in the discussion thread.

1160 implements pluggable transformers.  The swc transformer is built-in, but anyone can write a plugin to use any other transformer. This means your existing TS project configuration is all you need to use a different transformer, and you get the same benefits as ts-node: sourcemap support, ESM loader, etc.

Also, for many people, transpileOnly is all they need.  Often they've made a configuration mistake: they believe transpileOnly is enabled, but it actually is not. We implemented #1243 to make it easier to debug configuration mistakes, enabling more people to use transpileOnly and to better understand their configuration.