Nx: Importing a publishable library into another publishable library fails

[wapplegate]

I've been working on a proof of concept using nx extensions. What I'd like to do is create multiple publishable libraries that will be hosted in an npm feed. In addition, the nx-workspace contains an app that would import these libraries.

So I have Lib1, Lib2, and Lib3. Lib1 contains core components + services. Lib2 imports Lib1 to use some of those dumb components. Finally, Lib2 is imported into the application in the workspace. Running the application and this all works correctly. I see my dumb components from Lib1 working correctly inside my Lib2 components. However, when I build Lib2 using "ng build Lib2" (which imports Lib1) to get it ready for publish on an npm feed, I get the following error:

File 'lib1.module.ts' is not under 'rootDir' 'lib2\src'. 'rootDir' is expected to contain all source files.

From doing research on the topic, it seems like this is an issue with ng-packgr potentially? Is importing one publishable library into another supported in an nx-workspace?

[thelgevold]

Hi. Can you please include a repro of this issue?

[wapplegate]

Sure thing. Just attached my actual project. There really aren't many modifications from the project that was generated. I added one application "huron-application" and 3 libraries "admin", "core", and "identity". The admin and core libraries build fine with ng build. The core library has some dumb components inside it which are imported and used by identity library. The identity.module.ts file imports the core module in order to use some of the dumb components. The identity library has one component "login-page" which is where the component from core is used. The identity library is then imported into the huron-application. If you run "ng serve huron-application", things work fine. Go to the /login url and you should see some text. If I try ng build identity you get the error I described above.
project.zip

[wapplegate]

I'm assuming what is happening is when the identity library is built, it needs to include the core library which has an alias in the tsconfig.json:

"paths": {
  "@huron/core": ["libs/core/src/index.ts"],
  "@huron/identity": ["libs/identity/src/index.ts"],
  "@huron/admin": ["libs/admin/src/index.ts"]
}

So, using the alias path of "huron/core" it goes outside the identity projects folder structure to get those files. I guess because it's outside the source directory of the identity library, ng-packagr doesn't know where to put them in relation to the identity library's source files. I would think this would be a supported scenario when using an nx workspace. Because I'd like to also host these 3 libraries on a private nuget feed.

[matheo]

I was able to build my libs using ng-packagr directly,
and passing a custom package.json with the external references like:

  "ngPackage": {
    "dest": "../../../../dist/libs/admin",
    "lib": {
      "entryFile": "index.ts",
      "umdModuleIds": {
        "@huron/core": "huron.core"
      }
    }
  }

I will be digging into the Angular build process with the bundled ng-packagr, because I saw that Nx removed the build command of the Angular CLI libraries.

[wapplegate]

@matheo Interesting, so you were able to build all 3 libraries, even the ones that import other libraries? I'm curious to know if the maintainers of nx feel my scenario is a legitimate one. Have you made any progress with your research? Thanks for taking the time to respond and check it out.

[matheo]

@Singularity222 currently my dep-graph looks like:

Soon I will add some testing utilities to the library and it will be widely used too.
I will be distributing the datepicker bundle in npm, the others will be private for the company.

Also, I'm using deep dependencies too like @coachcare/common/store enabling them in the tslint.json, and I would love an article from @vsavkin about best practices on libraries and enterprise repo architecture :)

[matheo]

@Singularity222 I cannot share the company repo but soon I would get some time to build a Nx playground repo and showup with this stuff. I still need to go deeper with the @angular-devkit/build-ng-packagr builder and contribute my findings back to nx ;)

[wapplegate]

Any maintainers have input on this issue? I want to introduce the extension into my company but the lack of any acknowledgment of this issue is a concern to me.

[matheo]

@Singularity222 we will figure this stuff together, don't worry
the main "problem" in a monorepo is versioning and tagging the libraries, what's your approach?

[wapplegate]

Well, I'm not really concerned about the versioning. The issue I was looking for some guidance on was using a publishable library within another publishable library. My plan was to have a few publishable libraries developers could use in their application. So they could use the "Core" library as a base for their Angular applications. This library would contain a lot of lower-level functionality components, like select menus and text-boxes. Then, if they wanted more advanced features, they could import the "Admin" or "Identity" libraries to get those extra features. However, because the "Core" library has these lower level components, I also wanted to use "Core" within "Admin" and "Identity". So that's the really what I've wanted to accomplish.

[matheo]

@Singularity222 I see, that's a weird architecture tho.
Right now I'm using ng-packagr secondary entrypoints and I'm exporting like:
@company/common exports the Angular Module
@company/common/components and @company/common/pipes as secondary exporters.
Perhaps they are in the same published library.

[wapplegate]

Not too familiar with that, but I'll do some research. Why do you think what I'm trying to do is weird? Legitimately asking. My thought process was the "Core" library would have a lot of general application components. Like buttons, tabs, cards and other dumb UI components. In addition to the "Core" library there would be other libraries with more advanced use cases. The "Identity" library would contain components and pages related to authentication, like a login page. Depending on the complexity of the application, a consumer of my libraries could bring in "Core" when they have a relatively simple application, or they can import "Core" and "Identity" for an application that is a bit more complex and requires user accounts.

Because the "Identity" library has a lot more advanced components, I wanted that library to import "Core" so it can use those buttons, tabs, and other styled components. Otherwise I have to program the same button twice in each one of my other libraries if I can't import "Core". See what I mean?

[matheo]

@Singularity222 Well, what I've learned building libraries is that your code must depend on sibling or child code, not from a parent. So your identity lib cannot be a child of core if it uses components or features of it.

I think you rather need many publishable libraries instead of one core with sublibraries that are publishable too. I'd suggest to code your Core library and use it inside your Angular App, so you will see when the compiler complains about circular dependencies and stuff, that you need to sort out with the correct code organizations and imports.

[wapplegate]

The "Core" library is imported into "Identity". "Identity" would not be imported into "Core" at all.

[matheo]

@Singularity222 then it's pretty easy, as I shown you, I have a dependency tree with many libraries, using common and api.

I will setup a demo repo this weekend hopefully, so you can see my solution manipulating ng-packagr directly ;)

[wapplegate]

Thanks I would appreciate that. I haven't been able to make it work myself.

[wapplegate]

So I was just able to get a publishable library to import and build another publishable library by passing a custom tsconfig.json file when building the importing library, and moving the built library files of the library you'd be importing into the node_modules folder. It looks like this is a technique used by folks using ng-packagr. It's a bit hacky, but I've tested most scenarios work. I'll post a little more detailed explanation when I have some time to write things up.

[MrCroft]

do you have some small sample repo with this working?
I just couldn't make it work, I've been all day at this. Wish it just worked out of the box. Hopefully, it will at some point.

@matheo I have no idea what "umdModuleIds" is for :)
@Singularity222 the approach you've found sounds better I believe, but I'm not sure how that custom tsconfig.json should look like, I just understand that it's for the importing library (ie the lib that imports stuff from other libs)

[wapplegate]

When I get back to my computer I’ll write something up for you. Probably
won’t get to it until sometime tonight. It’s not too complicated. I don’t
have a build process set-up, but I can show you how to get the importing
library to build correctly and you can go from there. I’m no expert by any
means, but it seems like people associated to the ng-packagr project have
used this method to overcome the error.

On Mon, Jul 9, 2018 at 6:38 PM Marian Stoica [e-mail]
wrote:

do you have some small sample repo with this working?
I just can't make it work, I've been all day at this.
@matheo https://github.com/matheo I have no idea what "umdModuleIds" is
for :)
@Singularity222 https://github.com/Singularity222 the approach you've
found sounds better I believe, but I'm not sure how that custom
tsconfig.json should look like, I just understand that it's for the
importing library (ie the lib that imports stuff from other libs)

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/nrwl/nx/issues/602#issuecomment-403642185, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AA7d2LCYw84vLcsaEDJMApLjU85nPoMFks5uE9t2gaJpZM4U0Yhk
.

[MrCroft]

@Singularity222 just a reminder, I don't think you've managed to find time yet.
Would it be easier if I do a sample repo on this?
Thanks!

[zack9433]

[UPDATED] I fixed this error, checkout this commit https://github.com/zack9433/poc-workspace/commit/7dfedf7fdaf852a64e3b55960ba0678438aedc64

and run yarn build tp-core && yarn build tp-common

=========================================================================

I have encountered this problem too. This is a sample repo

https://github.com/zack9433/poc-workspace

I create two library tp-common and tp-core both are publishable and tp-common library import tp-core. When I run yarn build tp-common, there is an error happened

[stefan-karlsson]

Thanks to @zack9433 's post i managed to fix this issue for my project.

Version

nrwl/nx: "6.2.0"

Steps taken

Add npm scope to libraries

libs/cognito/package.json:
"name": "@npmScope/cognito",

libs/authenticator/package.json:
"name": "@npmScope/authenticator",

Add dist lib path to parent lib in root tsconfig.json

 "paths": {
      "@npmScope/cognito": [
        "libs/cognito/src/index.ts"
      ],
      "@npmScope/authenticator": [
        "dist/libs/authenticator",
        "libs/authenticator/src/index.ts"
      ]
    }

[latchezar-kostov-ascendlearning-com]

@zack9433 @stefan-karlsson That seems to fix the build issue. However, when I serve my application, my dependency lib (e.g. tp-core) is not properly resolved by the lib that depends on it (e.g. tp-common). I have to delete the dist folder in order to get this to work.

[zack9433]

@latchezar-kostov-ascendlearning-com currently i add a prestart npm script to remove dist folder to avoid resolved issue for serve application.

[wapplegate]

Sorry I've been absent from this thread. Had some personal issues come up the past few weeks. The way I actually "solved" this issue (which may be stupid compared to what is described above) is for the libraries that had a dependency on another library, I modified the tsconfig to remove the dependent library as an alias (and there might be another change or two). So to the library that was importing it, it seemed like a third-party library that would be found in the node_modules directory. When I would build the library that imported another, I had a script that would build the imported library first, and then place it in the node_modules folder. I saw this solution suggested in an ng-packagr thread. It has worked for me without issue. The build process is more complicated, but using ng serve requires no other modifications to be made. I just have an npm script that handles the entire build process for all libraries, then I publish them to our private npm feed. Has worked pretty well for me so far.

[kotva006]

I am running into the same issue, even after using @stefan-karlsson fix it appears that nx isn't aware of the dependency between libraries.
I have created a sample project to demonstrate the issue
I have 2 libraries abc, and xyz. abc imports xyz, and building abc causes an error because xyz isn't built before abc.

Steps to reproduce (run either command from the root of the project dir):
ng build --project=abc
npm run affected:build -- --all

Both of them fail on building abc (though xyz will build when the second command is ran after abc fails to build)
I believe that the build is supposed to know which libraries are dependencies and build them first.

[mohyeid]

@vsavkin I think this is a very common case and it would be greatly appreciated to have your input into this. Thanks!

[vivekkumar27june88]

Any update or guideline from Nx team on how to proceed on this.

[marcopetreri]

Any news about this issue?

[fkolar]

I have the same issue so I wonder if there are any news on this

[wapplegate]

So what I've been doing for the past couple months has worked fine for me. It's a little hacky, but it works and I haven't had any problems that I'm aware of. I'm not an expert though. What I do for my libraries is I put a custom tsconfig file in the directory of my libraries that removes the aliases to the other libraries. I'm pretty sure I just copied the top level tsconfig in the workspace and just removed the alias portion. This stops the library that is dependent on another library from trying to import the library from the workspace and instead try to import the library as if it was any library that was downloaded to the node_modules folder. So if library B depends on library A, I build A and then copy the built files into the node_modules folder in a directory that matches your import statement: import { A } from "@Test/A", Then I build library B which depends on A. The modified tsconfig file I mentioned above is in the library B directory and should be used automatically when building B. This is the method I've been using to build 6 Angular libraries at my company. All libraries have seen production use. Haven't run into any issues yet. I'm working on an example workspace for you guys right now that should show it working.

[ankeshdave]

@Singularity222 It would be awesome if we can get a demo project or a stackblitz link 👌

[gund]

We are running in the same issue.

There are 2 publishable packages: core and layout

layout imports core and fails to build with same error.

Repo is public: https://github.com/orchestratora/orchestrator

Using scoped names does not help at all and copying build artifacts into node_modules does not look right.

I think input from NX maintainers is pretty much required here...

[gund]

Well I found a much nicer solution to this problem.

It boils down to TypeScript configuration tweaking. You basically have to add paths prop that will point to the built library you are trying to import.

In my case I had to add this to my /libs/layout/tsconfig.lib.json:

    "paths": {
      "@orchestrator/*": ["dist/libs/*"]
    }

It basically now allows me to import any library and TS will not complain.

One important caveat - you have to build library that you import before building this library.

And since NX understands dependencies graph I think it would be logical to add custom build command that would execute build of libraries in required order.

P.S.
Here is the commit with changes that made my libs build: https://github.com/orchestratora/orchestrator/pull/17/commits/350b3f5225d816e05bff08fa403d3fc48d5c4e83#diff-8b2cda563aeded49eb78a04ef3bb9c63L16

[conor-mac-aoidh]

I agree that NX should be capable of building the libraries in the order necessary based on the dep graph.

This also means, if I libA is dependent on libB, and I change libA, then nx affected:build should build libB then libA.

[gund]

@conor-mac-aoidh I think you meant if libA depends on libB, and you change libA, then only libA is built. But if you change libB then libB is built after which libA is built.

Visually:

Dep graph
libA => libB

Change: libA
Build: libA



md5-5d8f78db8478a2dfb22260e001971e8b



Change: libB
Build: 1.libB 2.libA

[conor-mac-aoidh]

@gund No, I mean if libA is built, libB needs to be built first.

This is a requirement iflibA is configured to build & publish with ng-packagr. I am using the setup suggested by @stefan-karlsson:

 "paths": {
      "@npmScope/libA": [
        "libs/libA/src/index.ts"
      ],
      "@npmScope/libB": [
        "dist/libs/libB",
        "libs/libB/src/index.ts"
      ]
    }

With this setup, if libA is only built, we end up with errors in the libA build such as:

error TS6059: File 'libs/libB/src/path/to/file' is not under 'rootDir' 'libs/libA/src'. 'rootDir' is expected to contain all source files.

The only way to resolve this, is to build libB first. As I am working in a large project with lots of libs (some are publishable, some aren't), it would be nice for npm run affected:build to build dependencies of publishable apps first. There may be some other way to work around this issue, but I haven't found any.

[LloydTDavies]

@gund
This seems to work for me when I build and try to serve but when I publish and try and consume my library through a different application It can't resolve the published library's dependency.

I've tried to add the dependent library into the include array of ts config but no cigar.

[fkolar]

Just pinging if there is any progress on this. The above solution worked fine until I upgraded to latest "@nrwl/nx": "^7.5.2",, where everything is different again and you get the same error.

'rootDir' is expected to contain all source files.....

Angular CLI: 7.3.1
Node: 10.9.0
OS: darwin x64
Angular: 7.2.4
... animations, common, compiler, compiler-cli, core, forms
... language-service, platform-browser, platform-browser-dynamic
... router

Package                            Version
------------------------------------------------------------
@angular-devkit/architect          0.13.1
@angular-devkit/build-angular      0.13.1
@angular-devkit/build-ng-packagr   0.13.1
@angular-devkit/build-optimizer    0.13.1
@angular-devkit/build-webpack      0.13.1
@angular-devkit/core               7.3.1
@angular-devkit/schematics         7.3.1
@angular/cli                       7.3.1
@ngtools/json-schema               1.1.0
@ngtools/webpack                   7.3.1
@schematics/angular                7.3.1
@schematics/update                 0.13.1
ng-packagr                         4.7.0
rxjs                               6.3.3
typescript                         3.2.4
webpack                            4.29.0

Maybe nx is used for simple project as nobody runs into this issue. But I still hope that have a library that depends on another library inside 1 mono repo is the most basic usecase.

reverting back to:

Angular CLI: 7.1.4
Node: 10.9.0
OS: darwin x64
Angular: 7.1.4
... animations, cli, common, compiler, compiler-cli, core, forms
... language-service, platform-browser, platform-browser-dynamic
... router

Package                            Version
------------------------------------------------------------
@angular-devkit/architect          0.11.4
@angular-devkit/build-angular      0.11.4
@angular-devkit/build-ng-packagr   0.11.4
@angular-devkit/build-optimizer    0.11.4
@angular-devkit/build-webpack      0.11.4
@angular-devkit/core               7.1.4
@angular-devkit/schematics         7.1.4
@angular/cdk                       7.2.0
@angular/material                  7.2.0
@ngtools/json-schema               1.1.0
@ngtools/webpack                   7.1.4
@schematics/angular                7.1.4
@schematics/update                 0.11.4
ng-packagr                         4.4.5
rxjs                               6.3.3
typescript                         3.1.6
webpack                            4.23.1

Where it still works . thanks for iny inputs.

[zack9433]

@fkolar i am encounter this problem after upgrading to v7.5.2 and i create a new tsconfig.lib.prod.json to overwrite paths only for production which means you have to add a configuration for building library.

{
  "extends": "../../tsconfig.json",
  "compilerOptions": {
    "outDir": "../../dist/out-tsc/libs/tp-common",
    "target": "es2015",
    "module": "es2015",
    "moduleResolution": "node",
    "declaration": true,
    "sourceMap": true,
    "inlineSources": true,
    "emitDecoratorMetadata": true,
    "experimentalDecorators": true,
    "importHelpers": true,
    "types": [],
    "lib": ["dom", "es2015"],
    "paths": {
      "@tp-ui/tp-core": ["dist/libs/tp-core"]
    }
  },
  "angularCompilerOptions": {
    "annotateForClosureCompiler": true,
    "skipTemplateCodegen": true,
    "strictMetadataEmit": true,
    "fullTemplateTypeCheck": true,
    "strictInjectionParameters": true,
    "flatModuleId": "AUTOGENERATED",
    "flatModuleOutFile": "AUTOGENERATED"
  },
  "exclude": ["src/test.ts", "**/*.spec.ts", "formly"]
}

angular.json

...

    "tp-common": {
      "root": "libs/tp-common",
      "sourceRoot": "libs/tp-common/src",
      "projectType": "library",
      "prefix": "tp",
      "architect": {
        "build": {
          "builder": "@angular-devkit/build-ng-packagr:build",
          "options": {
            "tsConfig": "libs/tp-common/tsconfig.lib.json",
            "project": "libs/tp-common/ng-package.json"
          },
          "configurations": {
            "production": {
              "tsConfig": "libs/tp-common/tsconfig.lib.prod.json",
              "project": "libs/tp-common/ng-package.prod.json"
            }
          }
        },
        "lint": {
          "builder": "@angular-devkit/build-angular:tslint",
          "options": {
            "tsConfig": [
              "libs/tp-common/tsconfig.lib.json",
              "libs/tp-common/tsconfig.spec.json",
              "libs/tp-common/tsconfig.spec.json"
            ],
            "exclude": ["**/node_modules/**"]
          }
        },
        "test": {
          "builder": "@nrwl/builders:jest",
          "options": {
            "jestConfig": "libs/tp-common/jest.config.js",
            "tsConfig": "libs/tp-common/tsconfig.spec.json",
            "setupFile": "libs/tp-common/src/test-setup.ts"
          }
        }
      }
    },
...

[fkolar]

thanks for the update, but let's hope there will be more system solution where each of us does not have to individual hecks ;-(

Are you on latest?
https://github.com/angular/angular-cli/issues/13683

Angular CLI: 7.3.1
Node: 10.9.0
OS: darwin x64
Angular: 7.2.4

[fkolar]

From Angular-CLI ticket => it’s a 3rd party issue and is being tracked here ng-packagr/ng-packagr#1210 and solved my problem !

[fallXone]

We also have this problem, and would love to hear from the NX team on this topic, as it seems like a legitimate use case. We have multiple libs that we still have to unfortunately publish right now. These libs depend on eachother, and when trying to build an individual library, it is compiling the main library and the other publishable libraries it depends on. This results in multiple folders outputted to the dist directory, which causes this:

The lib.js file is missing from the directory it is trying to build from, because it is actually creating two directories at this location, one for each library. I'm going to try the suggested fixes above, but I do have some concerns around them and if they are actually going to work or not.

[gund]

@fallXone I fixed exact issue by adding to each library's tsconfig.lib.json one line:

"paths": {
  "@yourscope/*": ["dist/libs/*"]
}

This will allow typescript to load your libs instead of compiling them. Also make sure you build them in order they depend on each other.

[fallXone]

@gund would this affect ng-serve though? Doesn't that mean those dist files would have to be there in order to serve up the app? I still want to make sure we can use ng-serve with live reload and such.

[gund]

@fallXone serve is not affected by this change, you will still be able to live reload from your source files. This is purely for building purposes.

[fallXone]

@gund this seems to lead me to another set of errors,

Warning: Can't resolve all parameters for .... many services within my package. Is the specified path relative to your workspace root?

[gund]

@fallXone yes my base is project root and all path's in lib configs are relative to that path.

[gund]

@siimveskilt I'm not sure why you quoted exactly mine comment, but we here discussing exactly the problem of this thread, which is Importing a publishable library into another publishable library fails.

And my comments were just a quick fixes to allow builds of imported libraries.

Obviously the steps that I've taken here have to be done automatically by NX schematic when you add publishable library, as well as provide a command to build all libs in order.

[rahulrsingh09]

@gund I still get the same error saying 'rootDir' is expected to contain all source files. and i am at the verge of tearing my a hairs apart

[devonsams]

The paths fixed worked for me, but I still have one problem:

Type definitions for inferred types are being rewritten like so:

// original
exporting$ = this.store.pipe(select(exportsQuery.getExporting));

// generated *.d.ts
exporting$: import("rxjs").Observable<import("../../../../../dist/libs/standard").KeyMap<boolean>>;

my tsconfig paths:

"paths": {
    "@frontend/standard": ["dist/libs/standard"]
}

I need it to generate like:

exporting$: import("rxjs").Observable<import("@frontend/standard").KeyMap<boolean>>;

Is there any way to achieve this?

my current work around is to import the types at the top of the file, even if they aren't directly referenced.

[fallXone]

I fixed this by using a combination of the exampels above. I had to declare the paths, but I had to reference two things

• the dist directory of the library
• the index.ts file of the library from src (not libs)

"paths": { "@suite/core": [ "dist/libs/core", "libs/core/src/index.ts" ] }

With a combination of these two things, i'm able to build the libraries.

[dbigintonpro]

Well I found a much nicer solution to this problem.

It boils down to TypeScript configuration tweaking. You basically have to add paths prop that will point to the built library you are trying to import.

In my case I had to add this to my /libs/layout/tsconfig.lib.json:

    "paths": {
      "@orchestrator/*": ["dist/libs/*"]
    }

It basically now allows me to import any library and TS will not complain.

One important caveat - you have to build library that you import before building this library.

And since NX understands dependencies graph I think it would be logical to add custom build command that would execute build of libraries in required order.

P.S.
Here is the commit with changes that made my libs build: orchestratora/orchestrator@350b3f5#diff-8b2cda563aeded49eb78a04ef3bb9c63L16

@gund Excellent. This worked for me 😄

[jeneg]

Hi! Any updates on this? I faced same issue and I believe it's pretty common use case.
I've created simple repo to reproduce this issue. Here I created 2 publishable libs and only thing I made by hand is imported LibBModule into LibAModule and build of LibAModule fails. I found that structure of dist folder for lib that imports other lib differs from lib without dependencies and that cause issue with not found file.
Solution proposed by @gund working but it requires additional steps and logic to build dependencies first.


@vsavkin, @FrozenPandaz can you guys please provide some support on this?

[viceice]

@fallXone I fixed exact issue by adding to each library's tsconfig.lib.json one line:

"paths": {
  "@yourscope/*": ["dist/libs/*"]
}

This will allow typescript to load your libs instead of compiling them. Also make sure you build them in order they depend on each other.

@gund Thanks for the working solution 🎉

[fimius23]

I also am running into this issue.
@vsavkin, @FrozenPandaz can you guys please provide some support on this?

[ffdc-test-user]

any news about this? seems like a very basic use case

[ardentia]

Can we expect an official fix for this?

[stewones]

going through the same =/

[siimveskilt]

If you are fine without parallel builds then set implicitDependencies in nx.json:

"lib1": {
      "tags": [],
      "implicitDependencies": ["lib2", "lib3"]
},

Have paths that map the library scope to the library location.
From source:

"paths": {
      "@some-scope/lib1": ["libs/lib1/src/index.ts"],
      "@some-scope/lib2": ["libs/lib2/src/index.ts"],
      "@some-scope/lib3": ["libs/lib3/src/index.ts"]
    }

or from dist (as was already mentioned in this thread):

"paths": {
      "@some-scope*": ["dist/libs/*"]
}

And use affected:build:
npm run affected:build -- --all

[un33k]

In order for ng-packager to build a publishable lib, it must ensure all dependencies are resolved properly. With that said, if some logic that is referenced by a publishable lib falls outside of the lib root directory, then, naturally it won't be able to include that logic directly into the npm package.

NX in its current form, is not meant for publishable packages that are not self-contained.

To use NX for all your publishable libs that have dependencies on one another, it would need an NX workspace per every publishable lib where the lib contains all its own logic and any other dependencies will have to come from the package.json as an npm package.

Best use of NX is as a mono-repo where builds are only done at app-level and/or at self-contained lib- level.

[conor-mac-aoidh]

After working around this issue for a few months, I wrote this script to build the libraries in the order they depend on eachother. I have done this by specifying the libraries as peerDependencies in a package.json for each library. Then the script will sort the libraries depending on whether they have dependencies on eachother.

This script works for me, but might need some adapting to a particular use case.

const { exec } = require('child_process');
const fs = require('fs');

const args = process.argv.splice(2, process.argv.length);
const affectedLibs = args[0].split(' ');

function execCmd(cmd) {
  return new Promise((resolve, reject) => {
    exec(cmd, (err, stderr, stdout) => {
      if (err) {
        return reject(err);
      }
      resolve(stderr.trim());
    });
  });
}

const libsDeps = affectedLibs.reduce((obj, lib) => {
  const packageFile = `./libs/${lib}/package.json`;
  if (fs.existsSync(packageFile)) {
    const packageDetails = JSON.parse(fs.readFileSync(packageFile, 'utf8'));
    obj[lib] = {
      name: packageDetails.name,
      deps: Object.keys(packageDetails.peerDependencies)
    };
  }
  return obj;
}, {});

const buildOrder = Object.keys(libsDeps).sort((a, b) => {
  const aDep = libsDeps[a], bDep = libsDeps[b];
  if (aDep.deps.includes(bDep.name)) {
    console.log(`${aDep.name} requires ${bDep.name}`);
    return 1;
  }
  if (bDep.deps.includes(aDep.name)) {
    console.log(`${bDep.name} requires ${aDep.name}`);
    return -1;
  }
  console.log(`${aDep.name} does not require ${bDep.name}`);
  return 1;
});

console.log(`Build order: ${buildOrder}`);

buildOrder.map(lib => () => execCmd(`ng build ${lib}`))
  .reduce((promiseChain, task) => promiseChain.then(() => task().then(console.log)), Promise.resolve([]))
  .then(() => console.log('Build completed!'))
  .catch(err => console.error(err) && process.exit(1));

It takes the output of nx affected:libs as an input:

$ nx affected:libs --base=develop
viewer shared lists form core
$ node bin/build-libs-in-order.js $(nx affected:libs --base=develop)
my-npm-scope/form requires my-npm-scope/shared
my-npm-scope/form does not require my-npm-scope/viewer
my-npm-scope/shared requires my-npm-scope/viewer
my-npm-scope/form does not require my-npm-scope/core
my-npm-scope/shared requires my-npm-scope/core
my-npm-scope/viewer requires my-npm-scope/core
Build order: core,viewer,shared,form
Building Angular Package
Building entry point 'my-npm-scope/core'
Compiling TypeScript sources through ngc
Bundling to FESM2015
Bundling to FESM5
Bundling to UMD
Minifying UMD bundle
Copying declaration files
Writing package metadata
Built my-npm-scope/core
Built Angular Package!
 - from: libs/core
 - to:   dist/libs/core
Building Angular Package
Building entry point 'my-npm-scope/viewer'
Compiling TypeScript sources through ngc
Bundling to FESM2015
Bundling to FESM5
Bundling to UMD
Minifying UMD bundle
Copying declaration files
Writing package metadata
Built my-npm-scope/viewer
Built Angular Package!
 - from: libs/viewer
 - to:   dist/libs/viewer
Building Angular Package
Building entry point 'my-npm-scope/shared'
Compiling TypeScript sources through ngc
Bundling to FESM2015
Bundling to FESM5
Bundling to UMD
Minifying UMD bundle
Copying declaration files
Writing package metadata
Built my-npm-scope/shared
Built Angular Package!
 - from: libs/shared
 - to:   dist/libs/shared
Building Angular Package
Building entry point 'my-npm-scope/form'
Compiling TypeScript sources through ngc
Bundling to FESM2015
Bundling to FESM5
Bundling to UMD
Minifying UMD bundle
Copying declaration files
Writing package metadata
Built my-npm-scope/form
Built Angular Package!
 - from: libs/form
 - to:   dist/libs/form

I hope this helps some people!

[rustygreen]

The ability to have a monorepo with the flexibility to also publish the libraries (which are not self-contained) for other uses is a really attractive solution (this is actually the reason I started using NX, only to find out it wasn't supported later). Is a solution for this being entertained or actively developed within NX?

[tcoz]

You can publish the librarie if you install ng-packagr. Their doc page says how to go about it (you add a package.json to the lib directory, then run ng packagr on it, after which you can use npm pack to create a locally usable file, or publish to npm).

I agree that it would be nice to just be able to ng build mylib, but the ng packagr solution is straightforward enough.

There are some gotchas, mostly having to do with “how do I build a lib that uses another one of my libs”, for which I’ve come up with a couple of workarounds (mostly just chaining commands or copying a resource into the intended lib), so no dealbreakers, but I agree that an nx integrated solution would be elegant and that nx does seem somewhat incomplete without it (although otherwise is an excellent addition to a vanilla CLI workspace).

On Aug 27, 2019, at 10:29 AM, Rusty Green [e-mail] wrote:

The ability to have a monorepo with the flexibility to also publish the libraries (which are not self-contained) for other uses is a really attractive solution (this is actually the reason I started using NX, only to find out it wasn't supported later). Is a solution for this being entertained or actively developed within NX?

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub https://github.com/nrwl/nx/issues/602?email_source=notifications&email_token=AAJIGE2U7MI6244VQQRHMFDQGU223A5CNFSM4FGRRBSKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD5H5YGQ#issuecomment-525327386, or mute the thread https://github.com/notifications/unsubscribe-auth/AAJIGE5T6EQOXZBY24ARPW3QGU223ANCNFSM4FGRRBSA.

[jstoppa]

I implemented this package dependencies in a plain package using Angular CLI, I was hoping Nrwl would handle this types of scenarios. Any chance this will get supported at some point?

[bmarti44]

My current solution for these issues (which is very similar to other people's workarounds here). The following instructions take into account that the following libraries, and their dependencies, exist.

A library data-models contains interfaces and objects used across libraries
A library ui-kit contains presentational components and consumes data-models
A library data-access contains services with biz logic and consumes data-models

1. Make sure the top level tsconfig.json is pointing to all of your barrel files for publishable libraries so everything works correctly while doing local development.
2. Publish your base library (in my case, data-models) to npm. This library does not depend on other libraries and should be able to be published.
3. Install your library that needs to be consumed in another library as a dependency (in my case, npm i @my-company/data-models in the package.json for my @my-company/data-access library
4. Update the tsconfig.lib.json path aliases to point at the npm published library (this will override what is in the global tsconfig.json. This should look something like "paths": { "@my-company/data-models": ["@my-company/data-models"] }
5. Now, you are no longer using the index.ts file that lives outside of the library's rootDir, you are using what you just published to npm. You should now be able to publish your library to npm without any problems (so do that).
6. After your library has been successfully published, delete the node_modules folder that was generated from installing the dependent library (in my case, the data-models library). This will prevent Jest from trying to use the locally installed version of the libraries in the local node_modules folder, and force it to use what you specified in the top level tsconfig.json file (and all your local development will still work).

There are some downsides to this, installing each lib as a dependency and using it in the lib's tsconfig, and installing it does seem a bit "hacky". But unfortunately, that seems to be the over arching theme in making this work... Also, if you have a circular dependency (two libs depending on each other) I'm not sure how well that would work (if at all). You would, at a minimum, have to stagger your changes to the libs.

This process should be easy enough to script out (right now I'm using a bunch of npm run scripts) but you could very easily loop over all of the package.json's, find all of the one's that are externally published (name is @my-company/something), find collect the dependencies list and sort the build and publish order by the order in which each library is dependent on each other.

I'm going to work on the script that does this, and see if there are any great ways to integrate better with nx.

Here's my current npm run script for building data-access as an example -> (cd libs/data-access && npm update && npm version patch) && ng build data-access && npm publish dist/libs/data-access && rm -rf libs/data-access/node_modules

And it's tsconfig.lib.json

{
  "extends": "./tsconfig.json",
  "compilerOptions": {
    "outDir": "../../dist/out-tsc/libs/data-access",
    "target": "es2015",
    "module": "es2015",
    "moduleResolution": "node",
    "declaration": true,
    "sourceMap": true,
    "inlineSources": true,
    "emitDecoratorMetadata": true,
    "experimentalDecorators": true,
    "importHelpers": true,
    "types": [],
    "lib": ["dom", "es2018"],
    "paths": {
      "@my-company/data-models": ["@my-company/data-models"]
    }
  },
  "angularCompilerOptions": {
    "annotateForClosureCompiler": true,
    "skipTemplateCodegen": true,
    "strictMetadataEmit": true,
    "fullTemplateTypeCheck": true,
    "strictInjectionParameters": true,
    "enableResourceInlining": true
  },
  "exclude": ["src/test.ts", "**/*.spec.ts"]
}

And my top level tsconfig.json

{
  "compileOnSave": false,
  "compilerOptions": {
    "sourceMap": true,
    "declaration": false,
    "moduleResolution": "node",
    "emitDecoratorMetadata": true,
    "experimentalDecorators": true,
    "importHelpers": true,
    "target": "es5",
    "module": "es2015",
    "typeRoots": ["node_modules/@types"],
    "lib": ["es2017", "dom"],
    "skipLibCheck": true,
    "skipDefaultLibCheck": true,
    "baseUrl": ".",
    "paths": {
      "@my-company/ui-kit": ["libs/ui-kit/src/index.ts"],
      "@my-company/data-models": ["libs/data-models/src/index.ts"],
      "@my-company/data-access": ["libs/data-access/src/index.ts"]
    }
  },
  "exclude": ["node_modules", "tmp", "**/testing/**"]
}

[nayfin]

I struggled with this for a couple days for a few reasons. The first was a stupid mistake, in my tsconfig.lib.json I didn't have my paths field nested inside of compileOptions, thus my root tsconfig path was never being overridden properly. After fixing this I started getting the super helpful ERROR: Cannot read property 'members' of undefined, which I finally realized was due to barreling all my components and directives in respective index.ts files and importing from those from the barrels in the library's main module. It worked fine in the ng serve context only when I tried to build these libraries individually did I get the error. I hope this is helpful to someone trying to do something similar.

[nadavsinai]

Were migrating a whole ecosystem of apps and libs from publishing to internal NPM server to a mono-repo, and obviously we're keeping the ability to publish, I have encountered this error and fixed it via introducing a tsconfig.lib.json to the workspace root folder, it includes the path aliases for all libs to be from the dist folder rather than the lib/ source. I then changed all the tsconfig.lib.json inside the libs to extend this tsconfig inplace of the regular one.
I adjusted the script by @conor-mac-aoidh a little so that libs build by reference order as described in the peerDependencies section of package.json
this works but is not ideal :

• paths in tsconfig.lib.json are to be manually created when creating a new lib
• all lib builds must occur (slow) and in order else it is easy to have source and compiled files out of sync.
  
  
  
  • it may be impossible to create a cyclic dependency between libs (unverified)
  
  

I guess it is about ngPackager changing the baseUrl/rootDir of tsc not allowing the compiler to work with .ts files outside the rootDir

[devinshoemaker]

I am facing this issue as well. I am currently attempting to build a publishable library that imports a component from a non-publishable library and renders it to a div as a way of making a small separate bundle that I bring into a legacy app. I have tried a couple of the path based workarounds, but nothing has worked for me so far, and publishing these builds seems overkill. Being able to get this working would be hugely beneficial.

[dalemanthei]

Thanks to all the commenters here and other related threads, we have been able to publish a component library based on Angular Material. Like Material we have components that depend on other components that needs to be built in sequence.

In our scenario each component (think button, checkbox, etc) lives in it's own nx lib. Each of our libs (components) is built using ng-packagr controlled by npm scripts in package.json. I have created a demo repo which puts this all together showing 3 components with dependencies on the build order.

Here's the simple demo repo to build a library that could be published and installed from @dmv/demo: nx-ng-packagr-demo

Our production repo has 2 applications and a pile of components. This is an example of the dep-graph when we were writing the library:

Maybe this can help other folks trying to solve a similar problem.

[vsavkin]

Thank you so much for your patience folks.

@juristr is working on reworking how publishable libraries work to better support this use case. The initial PR landed into master and will be a part of the 8.12 release.

[nayfin]

I've actually had a pretty good experience. I just made a silly mistake when configuring.

[gund]

Actually I've found a workaround for building libraries in correct order.

However I'm not sure that this was intended behavior - but it always works exactly how it needs to.
So the trick is to use run-many command to build all your libraries like so:

nx run-many --all --target build

This will execute build target in all your libraries and applications in the order they depend upon each other (at least this is how it was for me). I've tested this with clear setup on different machines and the order was always correct (dependencies wise).

@vsavkin maybe you can confirm if this is by design - or just a side-effect that we should not rely upon?

[conor-mac-aoidh]

@gund I've tried this command and it does not seem to take account of the dependencies. It does not build the publishable libs in the order they depend on each other.

@dalemanthei That's a similar use-case to me. I just checked out your repository and tried running nx build link and it throws this error:

➜  nx-ng-packagr-demo git:(master) nx build link
Building Angular Package

------------------------------------------------------------------------------
Building entry point '@dmv/demo/link'
------------------------------------------------------------------------------
Compiling TypeScript sources through ngc
ERROR: libs/link/src/lib/link.module.ts(4,28): error TS2307: Cannot find module '@dmv/demo/core'.

How are you building the libs in order in this repository? Is there a custom command that I need to run?

[dalemanthei]

@conor-mac-aoidh We use a custom set of npm scripts invoked using npm run build-libs. The sequencing of the component builds is done manually in the npm scripts.

I suspect the nx build link is dependent on nx understanding the build dependencies. In our use case the npm run dep-graph doesn't work due to issues between published package path and code paths in the project. So the dep-graph works if our components were @dmv/link, @dmv/button, etc. Introducing the demo into the path to produce @dmv/demo/button breaks the dep-graph.

I will try to get a real README up this morning. Over the weekend I was thinking it needs to outline the various adjustments to get the repo working. Hopefully the build-libs command will get you going for now.

edit: I updated the README to document the changes we made.

[devinshoemaker]

@vsavkin is there any documentation around this new feature? I read the release blog post, however, I am still having issues. After updating and running the migrations, I get this error when building my library that depends on another library in the workspace.

Bundling...
rpt2: options error TS6059: File '/Users/devin/code/workspace/libs/react-components/src/index.ts' is not under 'rootDir' '/Users/devin/code/workspace/libs/entry-points/long-query-alert/src'. 'rootDir' is expected to contain all source files.
rpt2: options error TS6059: File '/Users/devin/code/workspace/libs/react-components/src/lib/react-components.tsx' is not under 'rootDir' '/Users/devin/code/workspace/libs/entry-points/long-query-alert/src'. 'rootDir' is expected to contain all source files.
'@workspace/react-components' is imported by libs/entry-points/long-query-alert/src/index.ts, but could not be resolved – treating it as an external dependency
'@workspace/react-components' is imported by libs/entry-points/long-query-alert/src/index.ts, but could not be resolved – treating it as an external dependency
No name was provided for external module '@workspace/react-components' in output.globals – guessing 'reactComponents'

So, it did not alert me that my other library needed to be built first (despite deleting my dist/ directory beforehand), and it did not update any tsconfigs to fix the build. Is this user error?

[devinshoemaker]

I found this PR which indicates that this fix was only applied for Angular projects. I am currently using React and am facing the same issue.

[dalemanthei]

@vsavkin @juristr Do you think the work being done to build dependent libraries will help with my challenge of building an angular material like library?

I created a demo repo. Based on this issue I made a couple of variations on the folder structure to improve on the dep-graph generation.

Ideally we could build all the components using a single nx command vs the npm scripts we have now. Of more value is being able to use the dep-graph and affected scope commands which seems possible based on changing the folder structure of the project.

[juristr]

Hey all, building publishable libs that reference other publishable libs should be solved now and work as expected starting from our 8.12 release.

A couple of notes here:

• publishable libraries can only depend on other publishable libraries. There is a new option in the nx enforce boundaries lint rule that checks for it
• you will need to build your dependent library first. Say lib-a depends on lib-b, you need to first build lib-b and then lib-a. The reason is that when building publishable libraries we depend on the built outcome of dependent libraries, not from src. We think that better represents the real world usage and thus helps potentially uncover problems before you publish them to npm. We will improve the dev ergonomics here in that we will automatically build all dependencies in the future. For now you can use the “run-many” command

Let us know if you run into any issues :)

[juristr]

@dalemanthei I didn’t look into the details of your demo repo, but I think the Nx run-many command could help you out here.

[snebjorn]

I have a @nrwl/node:lib (publishable) package that I import into a @nrwl/angular:lib (publishable). When I run nx build I get this warning.

WARNING: No name was provided for external module '@nx-test/libtest' in output.globals – guessing 'libtest'

> nx run libtest:build
Building Angular Package
******************************************************************************
It is not recommended to publish Ivy libraries to NPM repositories.
Read more here: https://v9.angular.io/guide/ivy#maintaining-library-compatibility
******************************************************************************

------------------------------------------------------------------------------
Building entry point '@nx-test/libtest'
------------------------------------------------------------------------------
Compiling TypeScript sources through ngc
Bundling to FESM2015
Bundling to FESM5
Bundling to UMD
WARNING: No name was provided for external module '@nx-test/libtest' in output.globals – guessing 'libtest'
Minifying UMD bundle
Writing package metadata
Built @nx-test/libtest

------------------------------------------------------------------------------
Built Angular Package
 - from: C:\Users\FOO\Repos\nx-test\libs\libtest
 - to:   C:\Users\FOO\Repos\nx-test\dist\libs\libtest
------------------------------------------------------------------------------

[Mani2693]

@juristr thanks for the fix , I have one concern of building dependent library , This is because in Angular when we create a library that is non publishable we will not be able to build those libraries, That case we are forced to create a library as publishable because we need to build it. Hope you understand my concern

Recommendation: I would recommend nx to build an library without building dependent library, Or may be an option in nx.json takes care of connecting dependency libraries

Regards,
Mani

[pburgmer]

I have a workspace with multiple publishable libraries. E.g. B depends on A.
Building A works. Building B doesn't work. Nx always fails, even if i run it with --with-deps.

Some of the project B's dependencies have not been built yet. Please build these libraries before: A

[seaBubble]

I'm still seeing this issue with @nrwl/angular 9.5

I'm running
nx run lib2:build --with-deps --prod
where lib2 is a publishable lib that depends on another publishable lib

And getting
error TS6059: File '[...]/lib1/src/index.ts' is not under 'rootDir' '[...]/lib2/src'. 'rootDir' is expected to contain all source files.

lib1 is built successfully on its own.

Is there an extra step to make this work?

[samratshaw]

@vsavkin Do you have any guide for how to achieve this in React-based app which is using Nx?