Berry: The nearest package directory doesn't seem to be part of the project

[jjenzz]

Description

We have a monorepo with a package.json in the root that lists the workspaces:

"workspaces": [
    "packages/core/*",
    "packages/react/*",
],

I wanted to upgrade our version of Yarn so I removed all node_modules and our yarn.lock in root and tried yarn after the upgrade but got the following error:

Usage Error: The nearest package directory ({path}) doesn't seem to be part of the project declared in {path}.

This used to work fine with .yarn/releases/yarn-berry.js but not with .yarn/releases/yarn-2.4.0.cjs.

Solution

I created an empty yarn.lock in the root and tried yarn again and everything worked.

This was all a bit unintuitive. I'm not sure if maybe the error should suggest that as a solution or if it should just create one for me(?) but thought I'd raise for discussion.

[merceyz]

The error happens when you have a yarn.lock (or package.json) file higher up in the filetree, at that point Yarn thinks you're in a workspace you haven't declared and throws.

[larixer]

@merceyz I think that current behaviour can be improved too. Why we check for presence of yarn.lock at all, why don't we have the same algorithm as v1, why it was changed at all? Why can't we check upwards package.json and look if their workspace patterns cover CWD and if they don't - then don't consider them as a project root

[jjenzz]

The error happens when you have a yarn.lock (or package.json) file higher up in the filetree

Well I'll be blowed! It was moaning about my home directory which to my knowledge didn't have a yarn.lock in, but I just checked and there was a stray one there for some unknown reason 😅

I do wonder if there is a way to make this all a bit clearer. If nothing else, at least this issue is here for others to stumble across if they face the same error I guess.

[arcanis]

Why we check for presence of yarn.lock at all, why don't we have the same algorithm as v1, why it was changed at all? Why can't we check upwards package.json and look if their workspace patterns cover CWD and if they don't - then don't consider them as a project root

Because then packages installed in the n_m folders would have no way to be tracked back to the original project, since they necessarily wouldnt't be part of the workspaces field. Workarounding that would require special cases, and I want to avoid that.

I'm not sure we can make that clearer though ... we already say what's the problem, usually confusion only arise when users don't actually trust the message and think it's a bug 🤔

[jjenzz]

@arcanis It wasn't that I didn't trust the message, I just didn't understand what the message was telling me. I think a clearer message would be something like:

Usage Error: A `yarn.lock` file was found in {path} but the nearest package directory ({path}) doesn't seem to be part of that project.

- bullets...

If it told me I had a stray yarn.lock somewhere, I'd have found it 🙂

[arcanis]

We already do that, though - the full error message you should have includes the following bullet points:

• If the project directory is right, it might be that you forgot to list as a workspace.
• If it isn't, it's likely because you have a yarn.lock or package.json file there, confusing the project root detection

Cf here

[larixer]

Because then packages installed in the n_m folders would have no way to be tracked back to the original project, since they necessarily wouldnt't be part of the workspaces field. Workarounding that would require special cases, and I want to avoid that.

Which "special" cases? It was not a problem in v1, was it? The packages in the n_m folders are in n_m folders, we can detect that and take into account.

[arcanis]

Which "special" cases? It was not a problem in v1, was it? The packages in the n_m folders are in n_m folders, we can detect that and take into account.

The v1 had no linker concept, and special-casing node_modules folders is exactly the kind of special case I want to avoid. The problem here is the unexpected lockfile in the home directory (and in some level the error clarity), not the detection.

[merceyz]

We do have the option of ignoring the home directory and never use it as a project root

[jjenzz]

@arcanis I read all of that, and still didn't figure it out.

The error happens when you have a yarn.lock (or package.json) file higher up in the filetree

☝️ That made me realise what was going on so perhaps those exact words could be a bullet.

This was my thought process with the current bullets:

• If the project directory is right, it might be that you forgot to list as a workspace.

The project directory is right, maybe if I try adding "." as a workspace in package.json it will fix it? ... Nope.

• If it isn't, it's likely because you have a yarn.lock or package.json file there

Hmm, I don't have a yarn.lock in my monorepo root because I deleted it before running yarn and of course I have a package.json there, it's a workspace... I'm not sure what this bullet is expecting.

I think the key part is replacing "there" (where?) with "higher up the tree"

[larixer]

The v1 had no linker concept, and special-casing node_modules folders is exactly the kind of special case I want to avoid. The problem here is the unexpected lockfile in the home directory (and in some level the error clarity), not the detection.

The detection is a problem. It is not the only complaint, we see them again and again. And there were no such complaints for v1. No matter how well the error message will be done, the problem will not be solved, because it is not in the error message.

[arcanis]

This was my thought process with the current bullets:

• If the project directory is right, it might be that you forgot to list as a workspace.

The project directory is right, maybe if I try adding "." as a workspace in package.json it will fix it? ... Nope.

• If it isn't, it's likely because you have a yarn.lock or package.json file there

Hmm, I don't have a yarn.lock in my monorepo root because I deleted it before running yarn and of course I have a package.json there, it's a workspace... I'm not sure what this bullet is expecting.

Thanks, that's helpful! So this is what we printed:

The nearest package directory (/home/user/project) doesn't seem to be part of the project declared in /home/project.

- If the project directory is right, it might be that you forgot to list my-project as a workspace.
- If it isn't, it's likely because you have a yarn.lock or package.json file there, confusing the project root detection.

I think the key point from your message is that the project directory wasn't right. The error referenced /home/project as project directory, with /home/user/project being the package directory. However I can see that being confusing, the nuance is fairly subtle when you just want to use a random project.

Perhaps the following (plus colors, etc) would be clearer?

The nearest package directory (/home/user/my-project) doesn't seem to be part of what seems to be the project root in /home/project.

- If /home/project is the root of your project, it might be that you forgot to list my-project as a workspace.
- If it isn't, it's likely because you have a yarn.lock or package.json file there, confusing the project root detection.

We do have the option of ignoring the home directory and never use it as a project root

I'm not sure blocking it altogether in findProjectCwd is a good idea since it's called before each command, but we can at least check that before throwing the error, to make it even more to the point.

[larixer]

All of the above work can be avoided by changing detection from checking for yarn.lock to checking for presence of .yarn folder. This is although far from being perfect, will relieve the pain points with current detection and confusion it causes.

[arcanis]

All of the above work

The work in question is mostly doing a wording change. You're advocating completely replacing how the detection is made, which is more likely to have repercussions in unforeseen ways, and will merely hide the core issue - there's a lockfile up in the tree, which isn't supposed to happen. As a general rule, when the fix is reasonable (and it is here, the question is mostly to convey it properly to the user), I prefer Yarn to abort early if something is wrong rather than try to make it work unchecked.

[larixer]

and will merely hide the core issue - there's a lockfile up in the tree, which isn't supposed to happen

I don't see it as a core issue. The presence of a lockfile which is NOT part of a project, should not be an issue to Yarn. I see the current behaviour as a false positive, Yarn misses real project root, leaves the project boundary while going upwards and finds the project root at the wrong place, this is the core issue.

[larixer]

Maybe clearer expression of my idea is:
Yarn 2 tries to exercise control outside of the current project upwards, without boundaries, I think it is not right. Yarn should exercise control inside project boundaries and it should not bother Yarn what is upwards in directory hierarchy, if there is yarn.lock, package.json or anything else upwards in the directory hierarchy. In order to exercise control within project boundaries Yarn should not miss real project boundary and go upwards past it, how to do it right is a separate question.

[jjenzz]

Perhaps the following (plus colors, etc) would be clearer?

This is spot on 👌 much clearer 🙂

[yarnbot]

Hi! 👋

This issue looks stale, and doesn't feature the reproducible label - which implies that you didn't provide a working reproduction using Sherlock. As a result, it'll be closed in a few days unless a maintainer explicitly vouches for it or you edit your first post to include a formal reproduction (you can use the playground for that).

Note that we require Sherlock reproductions for long-lived issues (rather than standalone git repositories or similar) because we're a small team. Sherlock gives us the ability to check which bugs are still affecting the master branch at any given point, and decreases the amount of code we need to run on our own machines (thus leading to faster bug resolutions). It helps us help you! 😃

If you absolutely cannot reproduce a bug on Sherlock (for example because it's a Windows-only issue), a maintainer will have to manually add the upholded label. Thanks for helping us triaging our repository! 🌟