Parcel: Why is node_modules watched?

Created on 16 Jun 2018 · 13Comments · Source: parcel-bundler/parcel

❔ Question

Why is node_modules watched?

Intuitively, only source files defined by the user should be watched.

🔦 Context

Errors are cropping up of the form

"Error: ENOSPC: no space left on device, watch 'my/project/path/node_modules/some/random/file.js'

I'm experiencing this error, and it's also been reported in #1438 and #1427.

It appears that every file in node_modules (recursively) is being watched for changes. Intuitively, this should not be necessary. Only the source files should be watched. Right?

Is there any way to disable watching all of node_modules? It is possible to tell Chokidar not to watch node_modules.

💻 Code Sample

index.html:

<html>
<head>
    <title>Parcel Example</title>
    <meta charset="UTF-8" />
</head>
<body>
    <div id="app"></div>
    <script src="src/index.js"></script>
</body>
</html>

src/index.js:

console.log('hello Parcel');

🌍 Your Environment

Ubuntu

| Software | Version(s) |
| ---------------- | ---------- |
| Parcel |1.9.0
| Node |v9.11.1
| npm/Yarn |6.0.0
| Operating System |Ubuntu 17.10

Source

curran

Most helpful comment

@devongovett Thanks for clarifying your stance here. Super useful to know that it is intentional. The symlinked node_modules is a use case I hadn't considered for this, also editing files in node_modules.

That said, it still does cause Error: ENOSPC, which may be the first experience folks have when evaluating Parcel (at least this was my experience - first thing an error, out of the box).

Also, I wonder what percentage of Parcel users do use symlinked node_modules or have a habit of editing files node_modules. My gut feeling is that this would be a small percentage.

Would you be open to considering adding a command line option to disable watching node_modules?

curran on 18 Jun 2018

👍8

All 13 comments

Not sure I've opened up a PR to ignore node_modules

DeMoorJasper on 16 Jun 2018

Side note, I was surprised too to see the node_modules being watched, in a positive way. I could directly start debugging the internals of a transitive dependency. If the CPU/memory overhead is negligible, I'm all 💯 for keeping it.

oliviertassinari on 17 Jun 2018

I'm against keeping it; if parcel-bundler is in devDependencies, this easily causes ENOSPC. It shouldn't be necessary to modify a sysctl to get this to work...

remexre on 18 Jun 2018

👍2

yeah watching node_modules is pretty useful, especially when you have things in there that are symlinked for local development. I suppose we could detect symlinked things and watch those (using the realpath). Also as @oliviertassinari sometimes its really useful to be able to edit a file in node_modules to test something out.

devongovett on 18 Jun 2018

That said, it still does cause Error: ENOSPC, which may be the first experience folks have when evaluating Parcel (at least this was my experience - first thing an error, out of the box).

Also, I wonder what percentage of Parcel users do use symlinked node_modules or have a habit of editing files node_modules. My gut feeling is that this would be a small percentage.

Would you be open to considering adding a command line option to disable watching node_modules?

curran on 18 Jun 2018

👍8

I agree with @curran.

I'm trying Parcel for the first time, and when I ran parcel index.html, I had a bunch of ENOSPC errors. A little bit of search led me here, and I see that this is because node_modules directory is recursively watched. This is OK, but maybe there should be a warning in the documentation explaining this and linking to a solution (which seems to be increasing fs.inotify.max_user_watches to some random very high value).

drazik on 24 Jun 2018

For me this sounds like really weird decision which will make Parcel lose completely to alternatives in the long run. To rephrase: in order to retain quite rare behaviour of reloading files in node_modules you are telling people to copy and paste a system-changing command from the internet. And even better, it starts with sudo.

While I know what it does and how its security looks, most people does not have to. And they actually should not like being forced to change their system for Parcel to work.

Watching node_modules should be an opt-in behaviour in my opinion.

katafrakt on 24 Jun 2018

👍6

I have a quick fix for this, add this.shouldWatchDirs = false

onparcel-bundler/src/Watcher.js. I don't know why it needs to watch dir though (and all things inside), watching individual files should be fine, I debug which files are watched after above patch, the result is exactly the files needed. Before the patch, parcel watches all the files under node_modules which is really unnecessary and was the cause for error.

reed1 on 27 Jun 2018

👍2

Thanks for sharing the workaround, @reed1. I'm guessing adding node_modules to ignored would work too.

katafrakt on 27 Jun 2018

curran on 28 Jun 2018

I'd like to relate a story.

A friend and I started a JS project, and I recommended Parcel. As expected, this ENOSPC error is the first thing he saw. Then I said oh yes, all you need to do to fix that is sudo gedit /etc/sysctl.conf then add the line fs.inotify.max_user_watches=524288.

At this point, he ran parcel, and his computer crashed. Our video chat got cut off.

After a few minutes he came back online and said something like, oh man ever since installing Parcel my machine has been acting weird. The WIFI stopped working, and the CPU overheats, and the system just freezes up when I run parcel serve. Then he asked "How can I get my system back to the way it was before I installed Parcel?". I recommended to remove that line in the /etc/sysctl.conf file, then rm -rf node_modules in both the home directory and project directory, to reclaim disk space.

He's got an older machine with only 4GB RAM. My guess is that the watchers were consuming all of his kernel memory, which was causing the OS to go down.

Each file watch takes up 540 bytes (32-bit) or ~1kB (64-bit) (according to this StackOverflow article). And how many files are there in node_modules? The result of find ./node_modules/ | wc -l is 13855. So around 13MB of kernel memory is taken up by watchers. Maybe it doesn't sound like much, but this is _kernel_ memory, not user space memory.

Bottom line is, we're dropping Parcel and will use Rollup instead for bundling modules.

curran on 28 Jun 2018

👍1

Hi @devongovett

This bug seems to qualify as a regression, but no fix has been released since it got reported here. The bug report had been closed before a PR was even considered. A few days after, PR #1559 was proposed but is kind of lingering, and it may be so because it's not very clear what the definitive stand on the matter is:

Parcel 1.9.x made a decision to watch node_modules by default (which breaks things)
yet you said it's more of an edge-case, a quite useful one, but probably not the main use-case (quoting you: "watching node_modules is pretty useful, especially when you have things in there that are symlinked for local development. […] sometimes its really useful to be able to edit a file in node_modules to test something out.")
PR #1559 would revert the default behaviour to not watching node_modules, and it seems like you would agree merging it, but it does not seem to provide a way to opt-in for watching modules (globally or through symlinks)

On the short term, would it be possible to:

clarify what Parcel's position is on the matter (is watching node_modules considered a default behaviour, for the time being and in the forthcoming releases?)
provide an opt-out mechanism (for instance an ENV variable)

Thank you! :)