Yarn: Exclude node_modules folders from Spotlight indexing

[as-com]

Do you want to request a feature or report a bug?
Feature

What is the current behavior?
node_modules folders are indexed by Spotlight on macOS

What is the expected behavior?
node_modules folders are (possibly optionally) excluded from indexing

This can be done by putting a .metadata_never_index in node_modules whenever the folder is created.

This may not work (it didn't work for me?)
Another way is to create node_modules.noindex, and symlink node_modules to it. Alternatively, figure out a way to programmatically add to the list of Spotlight exclusions. Looks like I was holding it wrong.

[burkeshartsis]

It seems like this only just became an issue when updating macOS to Mojave 🤷‍♂️

[as-com]

@burkeshartsis I thought node_modules polluting spotlight search results and high CPU usage from indexing always been an issue.

[tcrossland]

See also https://github.com/npm/npm/issues/15346

[jrz]

15346 says the "true" solution is add folder names to the index. This is pure nonsense. It should be possible to add the metadata_never_index, OR to set a file attribute.
It's either this, or having a global repository and better module loading (and only have spotlight index the files once).

[chrisbull]

* UPDATE * .metadata_never_index doesn't work on Majave or Catalina

Found this quick script to add .metadat_never_index to your node_modules folders.

find . -type d -name "node_modules" -exec touch "{}/.metadata_never_index" \;

I added it as a bash alias, so I can use apply quickly.

alias fix-spotlight='find . -type d -name "node_modules" -exec touch "{}/.metadata_never_index" \;'

Just thought I'd share here, for anyone else that is looking. :)

[kiddyfurby]

I am sorry but will yarn consider this feature?

[dicksonhk]

Based on @chrisbull 's idea, this script does the same thing faster by not going inside any node_modules directories found (coz the whole node_modules directory had already been ignored)

find . -type d  -path '*node_modules/*' -prune -o -type d -name 'node_modules' -exec touch '{}/.metadata_never_index' \;

For myself, I run it on the home directory, so I exclude hidden directories and Library directory to speed things up

find . -type d -path './.*' -prune -o -path './Pictures*' -prune -o -path './Library*' -prune -o -path '*node_modules/*' -prune -o -type d -name 'node_modules' -exec touch '{}/.metadata_never_index' \; -print

I also use the -print option so I know which folder is affected

[justin808]

For those using @dicksonhk's answer (thanks!) on MacOS, you have to have a dot or other directory after the command "find".

Maybe the same for other linux.

I agree that yarn should automatically add this file, although this is MacOS specific. Maybe some hook is better?

[dicksonhk]

For those using @dicksonhk's answer (thanks!) on MacOS, you have to have a dot or other directory after the command "find".

Maybe the same for other linux.

I agree that yarn should automatically add this file, although this is MacOS specific. Maybe some hook is better?

@justin808 You're right! I edited my answer to add . -type d and it should work fine!
Also, if you are running the command in the home directory like me you should take a look at the second command which excludes unnecessary directories :)

[joaomnb]

My Spotlight seems to ignore these .metadata_never_index files, every node_modules folder is still indexed... I'm running macOS Mojave. Anyone had the same problem?

[rolandleth]

Until this gets added — if it will — another approach would be to add a postinstall script:

"postinstall": "touch ./node_modules/.metadata_never_index"

Also, I didn't manage to make @dicksonhk's script to work (maybe because I'm using fish, dunno), but I did find another handy little script: https://github.com/Willian-Zhang/Ignore-node_modules-in-Spotlight.

[0x80]

@rolandleth I think it makes more sense to use "preinstall", so spotlight doesn't start picking up the files while the install process is ongoing.

But now I tried it is seems that postinstall and preinstall both don't add the file to my node_modules. Even if I delete the folder and run install again 🤔

[rolandleth]

There is no node_modules folder at preinstall, though, if you come from a clean slate.

Be sure to show hidden files when checking if it's there or not:

[c-bandy]

@rolandleth

There is no node_modules folder at preinstall, though, if you come from a clean slate.

"preinstall": "mkdir -p node_modules; touch ./node_modules/.metadata_never_index"

[rolandleth]

Ha, nice call. Sadly, the file gets deleted after yarn finishes, no idea why 🤷🏻‍♂️

[MariuzM]

Guys does adding .metadata_never_index really work? Because it's still indexing node_modules and subfolders. I've ran
sudo find . -type d -path './.*' -prune -o -path './Pictures*' -prune -o -path './Library*' -prune -o -path '*node_modules/*' -prune -o -type d -name 'node_modules' -exec touch '{}/.metadata_never_index' \; -print
And then I cleared my index by using

sudo mdutil -i off /
sudo rm -rf /.Spotlight*
sudo mdutil -i on /
sudo mdutil -E /

And noticed that it still indexing node folders/subfolders

[shanedg]

I didn't see any change with .metadata_never_index (caveat that I don't think I cleared my index after adding) but adding my projects folders to to the Spotlight privacy list does seem to have helped. I haven't noticed high resource usage by Spotlight processes since (Mojave and Catalina).

[joaomnb]

Same. As I had said before, .metadata_never_index does not work on my system. It didn't with macOS Mojave, and still doesn't now with Catalina.

[tsujp]

I've read that .noindex might be an option for Spotlight? Do you see any results with that @joaomnb ?

[joaomnb]

No, doesn't work for me either. Spotlight still indexes everything...

[joaomnb]

I did some research on that .noindex thing and found this.

This works in a different way. If you want the folder test/ to be ignored by Spotlight, you can rename it to test.noindex/. While this does work, renaming folders is obviously not an acceptable solution...

[nemosmithasf]

.metadata_never_index doesn't work for me either on Catalina\

Potential workaround is to put all projects in a single folder, black-list that folder via spotlight privacy settings and then create a VScode workspace outside that folder which points to the project folder internally.

[Strajk]

I've glued together solutions found here and around the internets and automated it:

• Get current Spotlight Exclusions
• Find all node_modules dirs in a specified directory
• Add new ones to Spotlight Exclusions
• Restart system service to apply changes

It's definitely not a code that I would be proud of, but it works :)

https://github.com/Strajk/setup/blob/master/programs/prevent-spotlight-from-indexing-node-modules.js