Eslint-plugin-import: Eslint is slow when used with webpack

[mieszko4]

I am wondering if it is possible to boost up performance when using eslint-plugin-import with webpack.

In my project:

• I have 4500 modules (find node_modules/ -type f -name "package.json" | wc -l)
• I have 500 files that I am eslinting
• I use 8 aliases in webpack config resolve.alias
• I use 8 loader rules in webpack config module.rules.

Eslinting lasts ~30s, and I can see that import/no-unresolved takes most of the time:

Rule                              | Time (ms) | Relative
:---------------------------------|----------:|--------:
import/no-unresolved              |  4983.197 |    28.2%
react/prop-types                  |  1694.653 |     9.6%
react/sort-comp                   |  1321.233 |     7.5%
react/no-multi-comp               |  1264.966 |     7.2%
react/require-render-return       |  1221.941 |     6.9%
import/no-named-as-default        |   665.776 |     3.8%
react/no-deprecated               |   524.655 |     3.0%
indent                            |   261.081 |     1.5%
import/no-named-as-default-member |   248.505 |     1.4%
react/jsx-curly-spacing           |   198.665 |     1.1%

Any suggestions on improvement?

[ljharb]

If you disable the webpack aliases and the resolver config for them, does it go faster?

[mieszko4]

I found the problem!
I am using webpack.config.babel.js and the problem is that it reads and transpiles the entire webpack config each time it resolves the rule. And there are lots of calls, in my case over 150 of them.

I suggest the config is read once and then cached. I will make a pr.

[ljharb]

Glad you figured it out; seems like a webpack issue.

[ljharb]

Wait, or are you saying that eslint-plugin-import reads your webpack config?

[mieszko4]

Yes, eslint-import-resolver-webpack reads the config each time exports.resolve() is called.
The improvement could be that it reads it once and exports.resolve() calles the cached version.

[ljharb]

Sounds like a great improvement, thanks!

[mieszko4]

After some further investigation I concluded that caching it would not speed it up (too much) since require() that is used caches the call already. It is simply slow because of babel the first time it is called.
Each subsequent time it is quick as it uses the cached version already.

I've created a pr anyway as it moves out some common processing.

[mieszko4]

Actually, correction I was testing it wrong. With this pr I got an improvement of over 2s.

[benmosher]

fyi: caching it permanently will impair things like eslint-loader and eslint_d that hold things in memory for a long time. I use eslint_d for linting in Sublime to improve performance.

caching it impermanently (i.e. maybe for 30s tops) would be fine.

[ljharb]

Could it be cached permanently but with an md5 hash of the file's contents, so any change busts the cache?

[benmosher]

lol yep, just suggested that over on #789 😁

[mieszko4]

Hmm. using md5 of webpack config file will not make it quicker then.
This is because quite some time seems to be spent on resolving the webpack config file.
require used in the code caches the file that is required.
So at first pass it is slow because of babel, it caches the required webpack config, then at each subsequent pass it is slow because of file path resolution.

[benmosher]

ahhh, right, I forgot that every file doesn't necessarily have the same webpack config. part of the resolver is finding the "right" one. you're right, would need to do that every time anyway

[mieszko4]

I guess then that there is nothing much that can be done here, i.e. at least my pr (https://github.com/benmosher/eslint-plugin-import/pull/789) does not make sense.
Perhaps only giving an optional parameter to this rule, something like { cachable: true }

[benmosher]

fwiw, eslint_d substantially improves performance for me in normal (webpack) use. There are many caches that this plugin builds already, and eslint_d allows them to exist across invocations. (no help for CI but it's great for local use)

[mieszko4]

thanx @benmosher. I am using atom.io with https://github.com/Adezandee/fast-eslint and it works nice for me. I had a problem with https://github.com/AtomLinter/linter-eslint though, it was unuseably slow.
CI is a different story, but 5s longer build is fine.

[mikhailChibel]

@mieszko4 how did you get the table with rule execution times?

[mieszko4]

@mikhailChibel prefix the call with TIMING=1

[kamronbatman]

This is still a major problem. We have ./index.mjs that re-exports a component and I have a feeling that simply adding re-exports is making the linter exponentially slower.

[kamronbatman]

Tested 2.12.0 and it looks like I am having a different issue that is causing slowness.

[benmosher]

@kamronbatman 🤦‍♂️ ohh nooo! any ideas?

[benmosher]

so #1091 was driven by my own results dropping timing logs into the webpack resolver and looking at what took the most time; the stuff I cached reduced my times by 80%, though it was only ~5sec of my 60s total lint time and so it only shaved ~4s off.

if you could either use the Node profiler (though I found this difficult and of limited utility) or just add some timing captures and logs and see what regions of code specifically are slowing you down, that would be the most helpful. I have already fixed the big waste that I could see.

also: did you upgrade your webpack resolver to 0.10? most of the fixes (if not all of them) were in there, and not in v2.12 of the plugin.

[gfx]

In my production, import/cycle is extremely slow:

TIMING=true npx eslint --ext .ts,.tsx src

Rule | Time (ms) | Relative
:---------------------------------|----------:|--------:
import/no-cycle | 22167.840 | 62.9%
prettier/prettier | 8538.422 | 24.2%
@typescript-eslint/await-thenable | 1832.930 | 5.2%
react/no-deprecated | 553.444 | 1.6%
import/no-deprecated | 503.936 | 1.4%

It takes 22 seconds!

[ljharb]

@gfx the first rule will always be slow because it’s building up an import graph; see #1408 for typescript-specific slowness.

[gilbarbara]

I upgraded my webpack config to be a factory (exporting a function instead of a simple object like CRA does) and my ESLint timing went through the roof.

So instead of setting the webpack config factory directly to import/resolver, I'm using a file where I import the factory and export the result.

// webpack.config.eslint.js
const configFactory = require('./webpack.config');

module.exports = configFactory('development');

Before

Rule | Time (ms) | Relative
:-----------------------------------|----------:|--------:
import/no-unresolved | 82850.597 | 54.6%
import/no-named-as-default-member | 47993.517 | 31.6%
indent | 3229.228 | 2.1%
import/named | 3056.267 | 2.0%
react/void-dom-elements-no-children | 1000.009 | 0.7%
react/no-deprecated | 858.790 | 0.6%
react/jsx-no-bind | 784.099 | 0.5%
react/no-direct-mutation-state | 731.675 | 0.5%
react/destructuring-assignment | 593.046 | 0.4%
react/sort-comp | 483.998 | 0.3%

After

Rule | Time (ms) | Relative
:-----------------------------------|----------:|--------:
import/no-unresolved | 10543.458 | 32.5%
import/no-named-as-default-member | 3781.838 | 11.7%
indent | 2865.881 | 8.8%
react/void-dom-elements-no-children | 817.273 | 2.5%
react/no-deprecated | 810.339 | 2.5%
react/jsx-no-bind | 759.052 | 2.3%
react/no-direct-mutation-state | 696.619 | 2.1%
react/destructuring-assignment | 589.438 | 1.8%
import/named | 446.033 | 1.4%
react/no-typos | 416.777 | 1.3%