Yarn: install --focus tries to install workspace dependencies from npm

[jacobp100]

Do you want to request a feature or report a bug?

Report a bug

What is the current behavior?

I have a workspace setup with workspaces app, web and core. app and web depend on core

Running yarn install from app installs all dependencies, including workspace dependencies

Running yarn install --focus from app attempts to install web and core from npm.

What is the expected behavior?

It shouldn't be looking to npm at all.

Please mention your node.js, yarn and operating system version.

Yarn v1.7.0
Node v8.9.0

[bdwain]

Hi @jacobp100. If your packages are not on the npm registry. focus won't really work for you. The point is to pull down the pre-built version from a remote location. If they aren't stored in a remote location, focus can't really do anything different.

Out of curiosity, what behavior were you expecting from a focused install versus an unfocused one? Maybe the docs can be improved to make it more clear what focus can and can't do.

[jacobp100]

That makes sense! React native doesn't work too well with the symlinks, so I have a script to basically install the core module as if it was downloaded from npm (no symlinks etc). The script is pretty gnarly, so I was hoping I could get rid of it with this.

[bdwain]

ah i see. Have you seen nohoist? It was made specifically with react-native in mind and solves a lot of their symlink issues.

[jacobp100]

Yea that worked for all but the sibling workspace dependency. Thanks anyway! I’ll close because it looked like I just misunderstood this feature!

[jonaskello]

I got a similar problem when I created a new package and ran yarn install --foucs. It tried to install the package I was working on from npm. Since the package was brand new it had not yet been published to npm. I think it make sense that the dependencies has to be on the registry, but should the package you are working on have to be published too?

[bdwain]

Hi @jonaskello. Can you provide a a failing example? It should not request the package you are focusing on from npm. Only other packages in your project.

[jonaskello]

Actually, this was in a private monorepo which has scoped packages that publishes to a private on-premise registry (running verdaccio). But I suspect it would be the same if it would publish to to the public npm repository, I can try and see if I can reproduce in that case. Anyway, once I published the package once the error went away :-). Currently I'm instead stuck on the error in #5881.

[bdwain]

Hm. Regardless it should not have been an issue in the first place. Would you mind filing a separate issue from this and at-ing me?

[bdwain]

I think the bug causing your install to request your target workspace from npm was the same bug that was looking at the folder name rather than the package name. These should both be addressed in #5893

[mrm007]

I don't think the issue is solved. I've hit the same problem in our monorepo, where the structure looks like this:

thorium/
- package.json
- packages/
  - thorium-client/
    - package.json (scoped, published on npm)
  - thorium-server/
    - package.json (scoped, private: true, not published on npm)

Going to thorium-server and running yarn install --focus I expect Yarn would install all of its dependencies, but it complains about not finding thorium-server (itself) on the registry:

⌚ 13:20:48 thorium/packages/thorium-server on  master [$!] 📦 v0.5.0 ⬢ v9.11.1
➜ yarn install --focus
yarn install v1.7.0
[1/4] 🔍  Resolving packages...
error Couldn't find package "@myscope/[email protected]" required by "[email protected]" on the "npm" registry.
info Visit https://yarnpkg.com/en/docs/cli/install for documentation about this command.

There is no dependency in thorium-server on thorium-client or vice versa.

[bdwain]

@mrm007 is it possible you are being affected by #5881 ? there are issues with scoped packages in that one. I have a pr (#5893 ) to fix that but it has not been merged yet.

[mrm007]

@bdwain Yes, you're right. After carefully reading through #5893 and https://github.com/yarnpkg/yarn/issues/5864#issuecomment-392087043 it is indeed the same issue:

It tried to install the package I was working on from npm.

Thanks for taking care of it!

[bdwain]

cool. sorry about the bug. Hopefully we can get it merged in the next few days

[sberney]

The documentation says:

If you’re only making changes to a single workspace, use –focus to quickly install sibling dependencies from the registry rather than building all of them from scratch.

I took this to mean that --focus would quickly install sibling dependencies from the place where it put the sibling dependencies rather than building all of them from scratch.

True, it says registry. Maybe npm registry would be clearer, if less correct.

Turns out --focus doesn't fit my use case at all. I was hoping it was some magic to install the packages into the current directory in the manner I used to use to build my project. Like with file:../foo type references.

[bdwain]

@sberney I'd rather not say npm registry because it can be any registry. At work, I use it with an internal registry.

It's only meant to help avoid having to build all other workspaces when prebuilt copies can be installed from a remote location, which will usually be faster.

[HosseinAgha]

@bdwain I think the problem with focus and private packages is not fixed yet.
For the following structure:

- package.json
- packages
  - frontend 
     - package.json (this package is `private: true`)
  - backend
     - package.json (this package is `private: true`)
  - lib1
     - package.json (this is public and published to npm)

running yarn install --focus in backend throws the following error:

Couldn't find package "@myscope/[email protected]" required by "[email protected]"

even though no package depends on @myscope/frontend

[bdwain]

if they are all private why are you focusing? Aren't they not on the registry then?

There IS an issue with focus where every package needs to be on the registry, rather than just the dependencies of what you are focusing on. I'm not sure of the best way to fix this. The way I've gotten around it is to take the packages I don't want to be published and make them not private and instead say "files": [] or some short list of files so it doesn't publish everything. I know that's not ideal, but it's

[HosseinAgha]

No. only frontend and backend are private as they are not shipped by npm but they may depend on some other packages in the monorepo (like lib1) that are published to npm.
My use case is in a docker build pipeline. I'm trying to yarn install --production only the backend's dependencies and finally copy the backend folder (including its node_modules) to the next stage.
Adding nohoist is no use here as it is dependency based. Your isolated proposal may be better suited for this scenario.
I think the main issue is in yarn install. It should skip downloading private packages whether we use --focus or not.
Thanks for the response.

[bdwain]

It does skip downloading private packages without focus. With focus it does have that issue though.

In general I try to avoid focus in CI's anyway. I mainly viewed it as a convenience tool for development, though I guess it could also be used to shorten build times. But I like to be sure my whole repo builds for any given commit when building in a CI

[booninite]

There are a pretty big array of totally valid uses for yarn install --focus in CI and it's extremely counter-intuitive that it doesn't respect the private keyword in workspace package.json.

[nambrot]

I also wanted to point out that we unfortunately seem to be outside the intended use case for yarn install --focus but it would seem very useful. In our case we have a big monorepo that versions packages together, and we would love the ability to only install one package's dependency, without having to do so for all the siblings. But --focus seems to still want to install all dependencies.

[ttiurani]

I was bit by this inability to get only one package's node_modules (the server-side rendering nodejs part) independently created. To hack around this issue, I copied the package.json into a dedicated new work folder and also copied yarn.lock there, then ran yarn install --production --frozen-lockfile in that folder to generate a production-ready node_modules. Quite ugly, but the result should be correct.

[tyleo]

If I am understanding the problem correctly I believe I am encountering the same thing. I have an electron app and a website which share some common assets in a third workspace. All of these projects are private. The electron app and the website have additional dependencies which are also the same. I wanted to make sure that the electron app and the website could compile independently if they were separate projects not located under the root. Unfortunately, it was hard to verify whether the dependencies sections of my package.json files were correct because the dependencies would all be hoisted into the root directory.

At first I got around this by deleting one project or the other temporarily and building the remaining project in isolation. I have since discovered that I can install the node_modules separately in each folder using a package.json like:

"workspaces": {
    "packages": [
        "packages/*"
    ],
    "nohoist": [
        "**/*"
   ]
},

Unfortunately, this requires having the second package.json file and manually clearing all of my node_modules which is somewhat annoying. It would be great if this could be included as a command line to yarn install.