Nx: Nx 7.4 (and above) not resolving workspace library dependencies correctly

[igortg]

Expected Behavior

Upgrading from Nx 7.3 to 7.4 would not change nx affected:build --all build sequence

Current Behavior

Nx 7.4 (also tested with 7.5.1) changed the build sequence for nx affected:build --all and seems not taking in account workspace library dependencies. Angular build fails with error "The [lib-dependecy-element] could not be found". If I build the missing dependency using ng build , nx affected:build --all works just fine.

Failure Information (for bugs)

I'll try to create a simple environment to reproduce this, but wonder if there was some major change in dependency resolution from 7.3 to 7.4.

Context

Please provide any relevant information about your setup:

    "@angular-devkit/build-angular": "~0.10.0",
    "@angular-devkit/build-ng-packagr": "~0.10.0",
    "@angular/cli": "~7.1.0",

[vsavkin]

Sorry for the late reply. Could you provide a repro?

[tanepiper]

Hi @vsavkin I can provide a repro on this:

https://github.com/tanepiper/ngx-tinynodes/tree/release/v1.2.0

I moved my plugin architecture to be a separate module that is included in the monorepo. It's already published to NPM, but when I try build my main library it fails:

➜  ngx-tinynodes git:(release/v1.2.0) ✗ ng build ngx-editorjs        
Building Angular Package
Building entry point '@tinynodes/ngx-editorjs'
Compiling TypeScript sources through ngc
Bundling to FESM2015

BUILD ERROR
Could not resolve entry (/Users/tane/work/personal/ngx-tinynodes/dist/libs/ngx-editorjs/esm2015/tinynodes-ngx-editorjs.js)
Error: Could not resolve entry (/Users/tane/work/personal/ngx-tinynodes/dist/libs/ngx-editorjs/esm2015/tinynodes-ngx-editorjs.js)
    at error (/Users/tane/work/personal/ngx-tinynodes/node_modules/rollup/dist/rollup.js:3460:30)
    at /Users/tane/work/personal/ngx-tinynodes/node_modules/rollup/dist/rollup.js:21474:17
    at async Promise.all (index 0)

Could not resolve entry (/Users/tane/work/personal/ngx-tinynodes/dist/libs/ngx-editorjs/esm2015/tinynodes-ngx-editorjs.js)
Error: Could not resolve entry (/Users/tane/work/personal/ngx-tinynodes/dist/libs/ngx-editorjs/esm2015/tinynodes-ngx-editorjs.js)
    at error (/Users/tane/work/personal/ngx-tinynodes/node_modules/rollup/dist/rollup.js:3460:30)
    at /Users/tane/work/personal/ngx-tinynodes/node_modules/rollup/dist/rollup.js:21474:17
    at async Promise.all (index 0)

As you can see from the below screenshot through it the folder structure is different, it also builds the local dep and that results in another folder level.

Hopefully this can help you track down the bug as it seems to be affecting a few people.

[igortg]

I just solved the dependency resolution by changing the projects declaration order on angular.json. Is that supposed to be the solution? Or is your intent that Nx solves inter-libraries dependencies by itself @vsavkin?

[vsavkin]

@igortg the order of declarations in angular.json should not matter. It's probably a bug. Could you provide a minimal repro demonstrating the issue?

[vsavkin]

@tanepiper Hey! The provided link doesn't work. When I clone master and run ngx-editorjs, it builds successfully.

[AdamVig]

@vsavkin I ran into a similar issue. I have an Nx monorepo with no applications and with two published libraries, library-a and library-b. I added a third library, internal-lib, that I don't want to publish. library-b depends on internal-lib.

My build output is the following:

Running build for projects:
  library-b
Building Angular Package
Building entry point '@my-company/library-b'
Compiling TypeScript sources through ngc
Bundling to FESM2015

BUILD ERROR
Could not resolve entry (/Users/me/src/my-repo/dist/libs/library-b/esm2015/my-company-library-b.js)
Error: Could not resolve entry (/Users/me/src/my-repo/dist/libs/library-b/esm2015/my-company-library-b.js)
    at error (/Users/me/src/my-repo/node_modules/rollup/dist/rollup.js:3460:30)
    at /Users/me/src/my-repo/node_modules/rollup/dist/rollup.js:21474:17

Could not resolve entry (/Users/me/src/my-repo/dist/libs/library-b/esm2015/my-company-library-b.js)
Error: Could not resolve entry (/Users/me/src/my-repo/dist/libs/library-b/esm2015/my-company-library-b.js)
    at error (/Users/me/src/my-repo/node_modules/rollup/dist/rollup.js:3460:30)
    at /Users/me/src/my-repo/node_modules/rollup/dist/rollup.js:21474:17

I think this is an issue with paths. The `dist` folder for `library-a`, which does not depend on `internal-lib`, is the following:

tree dist/libs/library-a

dist/libs/library-a
├── README.md
├── bundles
│   ├── my-company-library-a.umd.js
│   ├── my-company-library-a.umd.js.map
│   ├── my-company-library-a.umd.min.js
│   └── my-company-library-a.umd.min.js.map
├── esm2015
│   ├── index.js
│   ├── index.ngsummary.json
│   ├── lib
│   │   ├── library-a.js
│   │   └── library-a.ngsummary.json
│   ├── my-company-library-a.js
│   └── my-company-library-a.ngsummary.json
├── esm5
│   ├── index.js
│   ├── lib
│   │   └── library-a.js
│   └── my-company-library-a.js
├── fesm2015
│   ├── my-company-library-a.js
│   └── my-company-library-a.js.map
├── fesm5
│   ├── my-company-library-a.js
│   └── my-company-library-a.js.map
├── index.d.ts
├── lib
│   └── library-a.d.ts
├── package.json
├── my-company-library-a.d.ts
└── my-company-library-a.metadata.json

But for library-b, which depends on internal-lib, the directory structure is different:

tree dist/libs/library-b

dist/libs/library-b
├── esm2015
│   ├── internal-lib
│   │   └── src
│   │       ├── if-true.js
│   │       ├── index.js
│   │       └── unless.js
│   └── library-b
│       └── src
│           ├── index.js
│           ├── index.ngsummary.json
│           ├── my-company-library-b.js
│           ├── my-company-library-b.ngsummary.json
│           ├── library-b.js
│           └── library-b.ngsummary.json
├── esm5
│   ├── internal-lib
│   │   └── src
│   │       ├── if-true.js
│   │       ├── index.js
│   │       └── unless.js
│   └── library-b
│       └── src
│           ├── index.js
│           ├── my-company-library-b.js
│           └── library-b.js
├── internal-lib
│   └── src
│       ├── if-true.d.ts
│       ├── index.d.ts
│       └── unless.d.ts
└── library-b
    └── src
        ├── index.d.ts
        ├── my-company-library-b.d.ts
        ├── my-company-library-b.metadata.json
        └── library-b.d.ts

Please let me know if I can provide any more information. If you feel you need a minimal reproduction to debug further, I can try to create one.

edit: this may be related to #930.

[tanepiper]

@tanepiper Hey! The provided link doesn't work. When I clone master and run ngx-editorjs, it builds successfully.

Hi there @vsavkin yes it does now as I don't ship the core with any plugins now using the plugin module as a hard dependency, it's now a peer dependency.

If you look at the commit here: https://github.com/tanepiper/ngx-tinynodes/commit/14d0cc3a9eb0c2c7f65a4431ac2da7d3f4ec06d4

Clone from this commit and you should see the issue.

[tanepiper]

@vsavkin @AdamVig I can confirm that #930 does seem to fix it, specifically, https://github.com/nrwl/nx/issues/930#issuecomment-480810046, where adding the dist path in the tsconfig worked.

[luchsamapparat]

Here's a simple repository to reproduce the issue:
https://github.com/luchsamapparat/nx-lib-dependency-poc

just run npm install and then ng build --project library-b

You can reproduce this issue by following these steps:

1. create a new Nx Workspace create-nx-workspace nx-lib-dependency-poc
2. create library-a as publishable ng g lib library-a --publishable
3. create library-b as publishable ng g lib library-b --publishable
4. import library-a in library-b, see ./libs/library-b/src/lib/library-b.module.ts
   (4.1 create an app as otherwise the angular dependencies would be missing ng g app demo)
5. build library-b ng build --project library-b

Building Angular Package
Building entry point '@poc/library-b'
Compiling TypeScript sources through ngc
Bundling to FESM2015

BUILD ERROR
Could not resolve entry ([...]\nx-lib-dependency-poc\dist\libs\library-b\esm2015\poc-library-b.js)
Error: Could not resolve entry ([...]\nx-lib-dependency-poc\dist\libs\library-b\esm2015\poc-library-b.js)
    at error ([...]\nx-lib-dependency-poc\node_modules\rollup\dist\rollup.js:3460:30)
    at [...]\nx-lib-dependency-poc\node_modules\rollup\dist\rollup.js:21474:17

Could not resolve entry ([...]\nx-lib-dependency-poc\dist\libs\library-b\esm2015\poc-library-b.js)
Error: Could not resolve entry ([...]\nx-lib-dependency-poc\dist\libs\library-b\esm2015\poc-library-b.js)
    at error ([...]\nx-lib-dependency-poc\node_modules\rollup\dist\rollup.js:3460:30)
    at [...]\nx-lib-dependency-poc\node_modules\rollup\dist\rollup.js:21474:17

When I do the same with a vanilla Angular workspace, this issue does not occur. As @tanepiper suggests, this is probably related to the path mapping in the tsconfig file.

[vsavkin]

Right, I think the simplest thing to make it work is to add the following:

    "paths": {
      "@issue1077/library-a": [
        "dist/libs/library-a",
        "libs/library-a/src/index.ts"
      ],
      "@issue1077/library-b": [
        "dist/libs/library-b",
        "libs/library-b/src/index.ts"
      ]
    }

affected:* command topologically sorts all the projects, so it will always build library-a before library-b.

[AdamVig]

@vsavkin The workaround you shared above does not work with the --parallel flag, for example nx affected:build --parallel.

It seems like it would be relatively straightforward (and relevant to the core offering of Nx) to make nx affected:* --parallel aware of the dependency graph, not building a project until all of that project's dependencies are built.

It would be nice to have a command like nx build <project>, too, that builds all of the project's dependencies before building the project itself.

[vsavkin]

Hey folks.

We did a lot of work on building multiple libraries depending on each other.

• affected:build will finally sort the projects and run what it can in parallel. This finally works reliably.
• When building buildable/publishable libraries, we will now always look at the dist (or whatever you compile the library to) to find the dependencies. So consuming a publishable library from the monorepo should be very similar to consuming it from node_modules.
• No need to update your tsconfig files. Everything happens automatically behind the scenes. We will also use the name provided the library's package.json to resolve the library cause that's what you would use if the library was published to NPM.
• This works really well in combination with caching (see here: https://blog.nrwl.io/distributed-caching-in-nx-164edfbc68e0)

Since so much has been improved, I'm going to close this issues. If you use Nx 8.12.7 or 9.0.2 and you have issues with buildable/publishable libraries, please open an issue, and we will take a look asap. This use case is very important for us.

[AdamVig]

@vsavkin Thanks for the update! I have been eagerly awaiting this. My current workaround in continuous integration has been the following:

affected="$(npx nx print-affected --with-deps)"
echo 'affected:' "$affected"

# Extract `projects` field and output in space-delimited format
projects="$(echo "$affected" | jq --raw-output '.projects | join(" ")')"
echo 'projects:' "$projects"

for project in $projects; do
    npx ng build --prod "$project"
done

As you can see, the projects are built serially, which is slow. I did always find it curious that the `print-affected` command printed the projects in the correct order but the `affected:build` command did not. In local development, our "workaround" is to run the build three or four times (once for each level of our dependency tree) to get everything built. I wanted to share the above in case it helps others! --- I updated to Nx 9.0.2 (see report below) and ran `npx nx affected:build`.

nx report

  @nrwl/angular : 9.0.2
  @nrwl/cli : 9.0.2
  @nrwl/cypress : 9.0.2
  @nrwl/eslint-plugin-nx : 9.0.2
  @nrwl/express : Not Found
  @nrwl/jest : 9.0.2
  @nrwl/linter : 9.0.2
  @nrwl/nest : Not Found
  @nrwl/next : Not Found
  @nrwl/node : Not Found
  @nrwl/react : 9.0.2
  @nrwl/schematics : Not Found
  @nrwl/tao : 9.0.2
  @nrwl/web : 9.0.2
  @nrwl/workspace : 9.0.2
  typescript : 3.7.5

Since I had changed many dependencies in package.json, the command attempted to build every project. Unfortunately, it _still_ did not build them in the correct order. It built one "base" project (many things depend on it, but it depends on nothing) first, but did not built the other "base" project until second-to-last.

I am happy to share my dependency graph and build output privately, if that would be helpful.