Yarn: How to use --modules-folder flag?

[borela]

Sorry if this was asked before but I couldn't find an example in the documentation/google search.

Does this flag mean that I could have a "vendor" directory instead of "node_modules"? How to use it?
I tried something like yarn install --modules-folder ./vendor/node_modules and it just installed the modules in ./node_modules.

Please mention your node.js, yarn and operating system version.
Windows 7
yarn 0.16.1

[dhruvdutt]

There's a bug actually. Make sure you delete the node_modules folder, then it should work fine.
If node_modules is present then it would always say Already up-to-date.

This thing isn't documented on the website. I'll do it within next couple of days.
Meanwhile, you can refer this.
https://github.com/yarnpkg/yarn/blob/master/src/cli/index.js#L61-L62

[borela]

Thank you @dhruvdutt it worked. Is there any way to set the new modules folder as default?

[Promixius]

a way to set the default module path would be favorable as running yarn upgrade later when a package has an update will again pull it into node_modules instead of ./vendor/node_modules

[dhruvdutt]

@Promixius I was thinking about the same. To make a configurable file or something for this purpose.

[nifgraup]

broke in 5300b482c851e2578ac1f3aa4908be4d0289dca8

[nifgraup]

any way to recover branch clean-workflow from #281?

[nifgraup]

I would be able to bisect this further if you push clean-workflow again @kittens

[nifgraup]

result of further bisecting:

There are only 'skip'ped commits left to test.
The first bad commit could be any of:
f5f77ed30fc1a0846305aa41645ab43ebdfdfa8a
61890e9e01e01af5c7fe782480f896bfa1d349e2
ec7b1e5f9e5dfa81f5e85f4491cf5d67f3bd8608
f13537fa9b60136d28095306d789756bdb4d9b8d
We cannot bisect more!

[nifgraup]

After working around skipped commits I found this to be the trouble maker:

commit ec7b1e5f9e5dfa81f5e85f4491cf5d67f3bd8608
Author: Sebastian McKenzie [email protected]
Date: Mon Sep 5 15:45:55 2016 +0100

Add integrity hash check to `kpm install`

[nsgundy]

If I run yarn install --production --modules-folder production to have a folder that can be deployed to the production environment and I then run yarn install to get a local node_modules folder to run the build with (in this particular order because of the bug mentioned here and addressed by the PR above), yarn removes all modules from the folder production. This can be worked around by renaming the folder before the second run of yarn install, but I would prefer if yarn would just ignore that folder as long as it isn't specified again using --modules-folder.
Could that be changed or is the current implementation on purpose and required?

[jodigiordano]

IMO --modules-folder should be available on all commands.
Simple use case:

yarn install --modules-folder tmp/modules
yarn run [command] --modules-folder tmp/modules

[pciavald]

specifying in package.json would be amazing. Any ETA on this ?
http://stackoverflow.com/questions/40488421/how-to-change-yarn-default-packages-directory

[bestander]

This is a great opportunity to contribute to the project.
Please feel free to send a PR with tests and we'll get it in

[StevenDoesStuffs]

Any progress? I'm not familiar enough with the yarn codebase, and this feature would be amazing!

[rcelha]

@jodigiordano If this API is accepted by the maintainers I'd implement it... it'd make my life easier

[JesseObrien]

In regards to https://github.com/yarnpkg/yarn/issues/1684#issuecomment-263764878, I'm curious as well. It seems that running yarn with --modules-folder will wipe out/modify any existing node_modules folders that yarn can see. Is this intended behaviour?

[jimtom2713]

We have also had this issue mentioned above and referenced by @JesseObrien. We use webpack to package code up and create our distribution folder. I notice that running the following command to generate production node_modules folder in our distribution directory deletes the other modules within other exisiting node_modules folders.

yarn install --production --modules-folder ./dist/node_modules

Is this intended? If so, what's the rationale behind removing other node_modules contents?

[bestander]

@jimtom2713, yes this is intended, Yarn guarantees that node_modules is clean every time you install.

[nsgundy]

@bestander would love to be able to run yarn install and yarn install --production --modules-folder ./dist/node_modules (or some new version of that) one after the other and as a result have all modules (both dev-dependencies and production-dependencies) in node_modules and only the production dependencies in ./dist/node_modules.

Currently that seems only possible if I rename the node_modules folder after running the first command and then renaming it back after running the second command.

[bestander]

Ah, I think it makes sense now.

So the --production run removes node_modules from CWD?

On Sat, 13 May 2017 at 00:46, Gundy [e-mail] wrote:

@bestander https://github.com/bestander would love to be able to run yarn
install and yarn install --production --modules-folder ./dist/node_modules
(or some new version of that) one after the other and as a result have all
modules (both dev-dependencies and production-dependencies) in
node_modules and only the production dependencies in ./dist/node_modules.

Currently that's only possible if you rename the node_modules folder
after running the first command and then renaming it back after running the
second command.

—
You are receiving this because you were mentioned.

Reply to this email directly, view it on GitHub
https://github.com/yarnpkg/yarn/issues/1684#issuecomment-301209308, or mute
the thread
https://github.com/notifications/unsubscribe-auth/ACBdWO0tVEKtZsp7jfqOrLXaw75T3TZgks5r5O86gaJpZM4KpeN8
.

[jimtom2713]

@bestander The behavior @nsgundy is looking for is similar to the one we have. We have webpack bundle and package the code into dist. Then we run yarn install --modules-folder ./dist/node_modules to install the distribution modules under the dist folder. However, this means that the modules in the current working directory are removed. In development, and staging grounds this wipes the development modules away and we've gotten around it by changing the name of the folder and the order of the testing steps, but it's a lot of extra steps. Also, in development boxes, testing the build process locally means that without those changes the node modules are removed and require a fresh install every time.

[nsgundy]

@bestander Yes, that's exactly what is happening.

[apwelsh]

IMHO.
The default of installing global node modules into the user's home directory is wrong, and illogical. Never should a system library (the result of a global install) be symlinked back to a user directory.
Enterprise users usually have a single home directory that is mounted to all the systems. This results in broken installs, and fickle environments. NPM has the npm_config_prefix environment variable that is used to install global packages into a truly global location (on that system).

As an example, I have a system script that uses npm-remote-ls. I run the script from more than one server. One the first server, I had installed the package, and launched my script. While it was running, on the second system Installed the package there, which crashed the first system's install, because the 2nd system's global location did not have the package installed, and so it downloaded, and installed it. however, the home directory, which is mounted via NFS had it in the yarn node_modules folder, which is where is really ran from, and this file was deleted, and reinstalled causing the crash.

If I define the npm_config_prefix to point to /local/path, then everything is localized to a per-system install. But with yarn, there is no way to do this w/o manually providing a command-line switch on each install.

I think yarn should honor the npm_config_prefix for more than just the bin directory's symlinks. It should also install yarn's node modules in this same location. Either in lib/node_modules along with npm for seamless integration, or into an alternate yarn folder under the npm_config_prefix folder. I tried using yarn_save_prefix env variable, but this is just the cache, not the node_modules folder. There does not appear to be an environment setting for relocating the node_modules folder.

[saper]

Looking at https://github.com/sass/node-sass/issues/2050, how useful is a directory that does not have node_modules in the name? Should this option be something like --modules-prefix instead with implicit node_modules added after this?

It seems that this is what most people wanted in npm (https://github.com/npm/npm/issues/775).

[schmunk42]

Is there a way to define the modules-folder in packages.json or via an ENV variable, but not manually every time on the command-line?

[BYK]

Is there a way to define the modules-folder in packages.json or via an ENV variable, but not manually every time on the command-line?

Not yet. I know you don't like it much when I say this but, honestly: would you like to try adding it? :)

[andreyantipov]

@BYK If you don't mind can i send PR with this feature? i think that feature may be important in many cases. For example, one of use case :) im using iCloud drive for working copy and node_modules stored in .tmp folder because there is no another way to put folder in ignore list in iCloud. Should it be presented in .yarnrc or package.json or both?

[BYK]

@AndreyAntipov totally fine. There is already a PR that tried to fix this which needs some updates so you can also take that over: #3792.

[hkdobrev]

I've put this in .yarnrc in the root of my project and it works:

--install.modules-folder app/node_modules

as per https://yarnpkg.com/en/docs/yarnrc

However, I'm encountering another issue. No matter if I use .yarnrc or --modules-folder on the command line I always get an empty node_modules folder in the root as well. Should I open a separate issue for this?

[zeroblaz3]

I'm having the same issue as @hkdobrev. I run yarn install --production --modules-folder app/vendor and my local node_modules folder gets wiped. The app folder is our production folder while everything above that is development. I would love an option to prevent the node_modules folder from being wiped.

[AndersDJohnson]

I'm having same issue, where node_modules in currently directory is emptied when running yarn install --production --modules-folder=....

[SMotaal]

A fun fact about .yarnrc syntax:

Instead of:

--add.modules-folder …
--install.modules-folder …
… …

Try:

--*.modules-folder …

@hkdobrev this does not result in a lingering node_modules although it might still be created and deleted instantaneously as yarn does its thing.

edit: Seems that yarn remove does not honour --*. the same as --remove. (so maybe other quirks)

[brunofin]

@smotaal How can I find all the available options for .yarnrc? It seems the options aren't documented anywhere.

Specifically, I can't find anything mentioning the modules folder options.

[SMotaal]

@brunofin That was just a fluke… Yes, there are hardly any breadcrumbs to follow (I think this is a sign of the times, everyone is doing that no-docs ≠ poor-docs thing now)

[BYK]

I think this is documented here: https://yarnpkg.com/en/docs/yarnrc#toc-cli-arguments

Basically, you can put any CLI flag into your yarnrc file, either as tehy are or prefixed with a yarn command to scope it to only that command.

[SMotaal]

@BYK Yeah, came across that but then --modules-folder in .yarnrc did not function per docs — would pre-bind a --modules-folder arg for all commands where it applies, but sadly for this arg the behaviour was not on par with others I used in prior occasions.

Whereas for some un* [documented, supported, intended, …] — can only assume its just the temporary useful kinda bug/hidden feature — using --*.modules-folder instead seems to do a little better than --modules-folder in a .yarnrc.

But even then, it seems that certain commands just seem too self-important and want to be mentioned individually as they don't fancy themselves being lumped with the rest of em :scream:

[BYK]

@smotaal that actually sounds like a bug. We already know that some commands which should honor --modules-folder does not. That said --*.whatever should be equivalent to --whatever and I'm pretty sure the code paths are either the same or very similar. It is here: https://github.com/yarnpkg/yarn/blob/f69cdda63ab6f36bdd21c6f4ab838e7b6200b1d6/src/rc.js#L30-L64

[SMotaal]

@BYK thats amazing, thanks!

My apologies, I would retest this to make certain but I cannot recall which project/context I encountered it for replicating... @brunofin do you think you're hitting this same inconsistency and if so, can you provide a recipe that is reproducible.

It is very clear from the code that commands involving buildRcArgs would be bound equally with --*.… as --… so if nothing has changed in this function since we last encountered that issue, I wonder if maybe there is a difference some point upstream or even downstream.

[brunofin]

What happened to me is quite specific.

We use Vagrant in our project for reasons, and previously we had node6/npm, but then I migrated the project to node9/Yarn, and later on, I found out that Yarn allows me to configure the node_modules folder location. This was great, because of a known problem with Vagrant and symbolic links that are created under the node_modules/.bin folder during linking phase this flag could help me solve it. The problem is that Vagrant/Windows won't allow symlinks in a network-shared folder, node then throws a protocol error and things break.

For this reason, I had my project set up in a weird way where package.json, bower.json and other files that should be in the root folder of the project were actually in a subfolder called config. The contents of this folder were linked to the root folder in the VM, all that so node_modules would not be a part of the network-shared folder but still in the root of the project, at least in the VM. Very ugly.

But with Yarn, I can configure the location of node_modules, so I changed it to be somewhere else in the VM, moved the config files to the root of the project where they belong, great.

Using --modules-folder ..., I still got modules downloaded to my project root node_modules folder.
So I used --*.modules-folder ... instead and the modules are now downloaded to the location I wanted them to be.

But still I get a node_modules folder in the root of my project when Yarn tries to create symbolic links to the installed modules' binaries, and that's the reason I am making those changes in the first place. I'm not sure if this is a bug, but I think node_modules/.bin symlinks should also respect the --*.modules-folder flag.

I solved my problem by also enabling --no-bin-links true in .yarnrc, that works except bower, gulp or any other module which has binaries won't work because they are not in PATH. Manually linking the binaries to a folder in PATH (like ~/bin) solves the problem for some modules, but Gulp, for example, won't accept that, claiming there is no local gulp installed.

It's a complicated situation a bit out of the scope of the talk, but that's why I asked @smotaal about the CLI documentation, in case I was missing some other command that could help me solve that situation. I just feel that not all the available commands are documented, or the documentation isn't clear enough about its flexibility and .yarnrc.

I also found out that Yarn won't respect the --production true flag in .yarnrc and will download all the devDependencies anyway.

[BYK]

@brunofin thanks for sharing your use case and experience. I was also struck by that symlink issue with Vagrant and Windows some time ago and I can definitely relate to that. --modules-folder and --*.modules-folder should do the same thing so not sure what's going on there. Rest, though, is probably related to this bug. We know that --modules-folder is not used everywhere where it should be used etc. and this bug is kept open to keep track of this.

I also found out that Yarn won't respect the --production true flag in .yarnrc and will download all the devDependencies anyway.

This also shouldn't be the case. You are either using an old version of yarn which has a bug regarding CLI options in .yarnrc which would explain the --*.modules-folder and --modules-folder difference above or there is something else going on with env variables or something. We have this whole test suite after that: https://github.com/yarnpkg/yarn/blob/f69cdda63ab6f36bdd21c6f4ab838e7b6200b1d6/__tests__/integration.js#L82-L127

[brunofin]

@BYK thanks a lot for clarifying. I'll make sure I'll check my versions, upgrade everything possible and try again. I'll let you know.

btw, any advice you can give me with the Vagrant situation? (sorry if I'm leading to an off-topic conversation).

[BYK]

@brunofin

btw, any advice you can give me with the Vagrant situation? (sorry if I'm leading to an off-topic conversation).

I think the newer versions of Windows allow the creation of symlinks without admin permissions: https://blogs.windows.com/buildingapps/2016/12/02/symlinks-windows-10/

If you want to chat more, come to our Discord server ;)

[miks]

What's the rationale behind --modules-folder flag if it's works only for install command?
Sure, I can install modules in specified folder, but that's all.
It's not working for example with run command.
Am I missing something?

[bdurrani]

I found this issue via this SO thread.

What if there are packages that you need to install for the server side (e.g. express) others that you need from browser side e.g. jquery, bootstrap, font-awesome?

I used NPM for server and bower for jquery, bootstrap etc ... now with Yarn alone I still don't know how to change destination for a particular library that I want in my assets/vendor folder instead or node_modules

[kwhitejr]

@nsgundy , @zeroblaz3 , @AndersDJohnson , @jimtom2713 : Anyone figure out a more satisfactory way of preventing the base level node_modules from being blown away when using yarn install --production --modules-folder other/directory/? Seems like the most straightforward workaround was to rename original node_modules before and after the yarn install?

[nsgundy]

@kwhitejr, unfortunately I have not.

[jimtom2713]

@kwhitejr I have not found a different way.

[chakradhar2]

I am using --modules-folder, which download's module for me in different folder. But not able to run the commands which I use to run before using --modules-folder, like gulp .
Is there any way, I can ask to look the modules in the new custome folder and do not have to change the all paths's in require() ?

[njlr]

@miks The reason I use the --modules-folder flag is to create a node_modules folder that contains only production dependencies, which are then bundled with my app. If I didn't do this then I would be bundling modules like babel, which bloats the binary.

[miks]

@njlr how you can run your code against custom node_modules folder after install?

[njlr]

@miks I bundle the whole thing (my code, node_modules and Node itself), for example using pkg.

[mohkale]

For some reason when I specify a modules folder, webpack crashes saying it Can't resolve 'babel-loader'. If i reset --modules-folder back to node_modules it works again. Does anyone else have this issue 😢.

This is weird. I've set my modules-folder path to ./vendor/node/. If I run yarn run webpack, it fails with the above error message. If I run NODE_PATH=./vendor/node/:$NODE_PATH node and then evaluate require('babel-loader') it runs fine. If I run NODE_PATH=./vendor/node/:$NODE_PATH yarn run webpack it doesn't.

[kaishin]

I am having the same issue as @mohkale.

[devinrhode2]

@kwhitejr in package.json "scripts":{"preinstall":"mv node_modules not_node_modules","postinstall":"mv not_node_modules node_modules"} should work in theory

[succerror]

Is the --modules-folder flag useful in anyway at all? Yes it allows to install packages in a different folder than node_modules, but what next?

Adding custom_folder/.bin to PATH and running the executables is not going to work(because the require statement in them is going to look for node_modules folder and will fail).
package.json scripts also has no way of knowing custom_folder is a thing, so yarn run will not use custom_folder.

There is no way yarn or npm can fix this because this is part of how node works.

There is no way in node to change node_modules to something else. It can only be moved to parent directory and sub-directories to isolate dependencies(please correct me if I'm wrong).

Edit:
NODE_PATH env variable can be set to tell node to search for packages also in custom_folder.
So the below combinations might work for some.

yarn install --modules-folder custom_folder
export PATH="$(pwd)/custom_folder/.bin:$PATH"
export NODE_PATH="$(pwd)/custom_folder:$NODE_PATH"

This might break bundlers where /node_modules/ is being matched, so consider using custom_folder/node_modules instead of just custom_folder.