Nx: Add ability to publish multiple libraries from a single nx workspace

[vlodko]

Multiple teams are working on different apps (multiple products within the same company) using Angular 2+.
Some of modules are shared between those apps.

Teams don't want to have single monorepo for all apps and libraries.
Each team would like to have monorepo for their app and libraries with ability to publish multiple NPM packages, so other teams could re-use them.

Each app/library should be developed on top of Nx/Angular CLI (No webpack scripts, make files for building app and libraries).

I'm new to Nx so if there a way to combine Nx with some existing tools like Yarn workspaces, Lerna or Rush, please, give me some directions.

Thanks

[kospiotr]

This is what I'm also looking for. I would like to develop a library with many modules core, common, comonents-basic, comonents-material, etc with demo site at once. Monorepo aproach is the correct way to. Currently I'm using lerna, however multiple node_modules are killing my IDE.

[vsavkin]

I wrote a blog post showing how to use Nx with Yarn workspaces and git submodules (lerna can be used in a similar way):

https://blog.nrwl.io/dev-workflow-using-git-submodules-and-yarn-workspaces-14fd06c07964

[vlodko]

thanks @vsavkin for a useful blog post.
now I need to play with it for some time :)

[noelmace]

In addition, there are many online resources on how to use Nx with ng-packagr for this kind of workflow. It's really new, so for now, I don't think there's any about "real / big" scenarios. I'm working on it for a big company, so I'll definitely write mine when it's done. Some examples for the moment:

• Create and publish Angular Libs like a pro by Juri Strumpflohner @ ngVikings 2018
• an example from the creator of nx-packagr
• ng-packagr
• PR for angular/devkit: this will certainly have a real impact on how we'll create angular libraries in the near future

Using git submodules like @vsavkin is also a really great option, but after testing it, it wasn't "the" good one for this projects I'm working on.

Linking : https://github.com/angular/devkit/pull/444

[noelmace]

Some feedbacks on my current work. We now have a working solution to manage "publishable" libraries in our Nx monorepo. I think this could lead to some PRs if @vsavkin and the other members of the nx core-team aggries - even if I'll need to work my schematics before anything ;)

Here is our solution :

1. Each "publishable" library have a specific package.json file (libs/*/package.json), which give its dependencies and peerDependencies, publishConfig and ngPackage configuration. This is how we detect that a library is publishable, that it (other libraries doesn't have any package.json file). By convention, the ngPackare.dest has to be "../../dist/npm-libs/" and the name "@/".
2. We use (with a "build:libs" npm script) a home made "build-libs" node script which detect this package.json files, and build each corresponding library with ng-packagr, but only if the "ensureConventions" method doesn't throw any exception for this lib. This method detect if :
   
   
   
   • ngPackage.dest is "../../dist//"
   
   
   • every dependencies (including peerDependencies) in this package.json file correspond to the ones in the root package.json file (in order to ensure that all the dependencies are coherent between projects)
   
   
3. another home made node script is responsible (when requested with the "publish" npm script) to detect builded libraries, and, if needed, to publish them to our npm repository.

So, this could lead to the following steps in order to had this to Nx :

1. add a new "npm-lib" schematic which would act as a final step to the "lib" one, and add the package.json file, or add an option to "lib" to do so
2. create a new script to manage ng-packagr libs build and publication
3. add a "--publishable" option to create-nx-workpace which would add the ng-packagr dependency and specific npm scripts

See also https://github.com/nrwl/nx/issues/309#issuecomment-376089674

@vsavkin or any core-team member, as soon as you can give me a validation on any of this points (and maybe some guidance), I'll start to work on a first PR.

[kylecordes]

BTW We do something similar here. The key feature in Nx that makes this possible, is that it is not bothered by the existence of package.json files in the lib directories. We using it was just a modest number of published libs, and so haven't needed the automatic machinery @noelmace is using, but something like that built into Nx would be fantastic.

[vlodko]

@noelmace thanks for the update!
Can you explain this:

By convention, the ngPackare.dest has to be "../../dist/npm-libs/" and the name "@/".

does it have anything to do with NPM scoped packages?

[noelmace]

@vlodko It's just some conventions we made for convenience in the named project. This way, we are sure that every builded lib is in the same parent folder and respect a convention that we could use in the other scripts and the CI.

Anyway, as this is a temporary solution (given thas we should either add this feature to nrwl/schematics, or publish another project using schematics) I'll negociate with my manager to publish this scripts on Github and npmjs.org tomorrow (european time). This would make this conversations easier, and avoid to flood nrwl/nx with not directly related topics.

[noelmace]

So it's done, this minimal CLI tool, now named packagr-for-nx we talked about is now available on github, and the 0.2.0 version should be published on npmjs.org within 24 hours. @vlodko this should answer your questions and avoid to polute this issue ;)

@vsavkin, I'll be happy to make a PR in order to permit to have this kind of feature in Nx, if you think this could be right. For now, I identified the following "missing" features :

• permit to generate a package.json with a default ng-packgr configuration when creating a new lib, using a specific option (for example, --pkg or --publishable). This would correspond to a new kind of lib, "publishable lib".
• add some "build" / "publish" script(s), like those in packagr-for-nx, to separatly build and publish to npm the "publishable libs"
• permit to bump the version in this package.json files, maybe when bumping the main one (I'm working on it for my project, but lerna wasn't a good option for that)
• maybe permit to add the ng-packgr (and any other required) dependency when creating a new workspace if a specific option is used

[tobi-or-not-tobi]

Great job @noelmace.

I'd like to emphasise the need of library packaging/publication support. I really like the mono-repo proposition of nrwl. But in order to reuse those libraries outside the workspace it would be required to package/publish individual libraries. The current setup is great for larger projects, but doesn't encourage them to share their libs.

I would need the following:

• ability to package/publish libraries individual. Libraries might follow their own release schedule. The beauty of the nrwl mono-repo approach is that apps would benefit from any updates on the library pre-publishing.
• avoid conflicting package.json files. I really like the single package.json file in the workspace, as it ensures all apps and libraries are aligned with the same dependencies. And the idea to add package.json for each and every library feels not right. However, we need to provide library info (dependencies, versioning, license, etc.) and package.json is the right place. If we could avoid any duplication or even conflicts, it would be nice.

I'd also be interesting to see how angular elements is going to influence this piece. I know angular elements is a labs project, but I believe it will get a lot of attention once its out. This will allow for more fine-grained publishing of smaller reusable (web) components – not sure if these would be wrapped insde libraries though. Once this is in place, we're able to mix and match components from different frameworks and ideally be able to user dynamic loading of components.

[noelmace]

By the way, angular-cli 6.0.0-rc.1 now support ng generate library thanks to the new angular schematics (https://github.com/angular/devkit/tree/master/packages/schematics/angular/library) added to angular-devkit by https://github.com/angular/devkit/pull/444.

I'm not for now really used to angular-devkit, but I think some consequences on this feature should be :

1. "publishable libraries" management in Nx will require an angular-cli v6.0.0 compatibility
2. this will require to rethink Nx ng generate lib entirely

This subjects are really bigger than this feature request only. I'm sure @vsavkin, @jeffbcross or any other core team member is currently working on this, or had at list thinked about it.

In fact, has ng g library generate a new library project in a projects folder, we could assume that this is an equivalent of Nx libs/ folder ...

[noelmace]

In order to address this issue, I'm currently working on Angular-cli v6 support, which means I also need to make the migration to angular 6, ngrx 6 and rxjs 6. I'm doing this because I'm now pretty sur that addressing this issue without using the new @schematics/angular "library" make no sence (and I'm not really a "wait and see" guy :smile:). This is a pretty big task, so @vsavkin, I would be really thankfull if you could give me any help or advice on this.

[tobi-or-not-tobi]

@noelmace FYI I was able to setup an app with embedded libraries in the latest RC, only had to work around one issue. See https://github.com/angular/angular-cli/issues/10369. It runs, updates while coding and builds both apps and libraries. Looks very promising.

[vsavkin]

This works in Nx 6.

Just create a library with --publishable flag, as follows: ng g lib mylib --publishable.

And run then ng build mylib

[dbigintonpro]

I'd like to see support for affected:build with a logical or defined order. There doesn't seem to be any ability to order the build process and this becomes a problem when having dependencies on other libraries.

[troywweber7]

This doesn't appear to work in Nx 8

It does work for the @nrwl/angular lib schematic, though. Is there a lib schematic for @nrwl/node that allows the node package to be publishable?

[alzalabany]

using "@nrwl/react": "9.1.2" i created new library using cmd

yarn nx g @nrwl/react:lib design --publishable

then i built it with yarn nx build design;

build success but when i inspect the dist folder, i found that no dependencies where added to package.json, although default template for react:lib using

Expected:

it should have included react, and styled-components in "peerDependencies"