Nest: webpack and bundle

[hkjeffchan]

[ ] Regression 
[ ] Bug report
[ ] Feature request
[x] Documentation issue or request
[ ] Support request => Please do not submit support request here, instead post your question on Stack Overflow.

I develop the frontend with angular and totally new to nestjs and nodejs. Would like to know if there is anything like "ng build --prod" to pack the whole project into the dist folder so that I can deploy it to the production server without installing all the node_modules?

What is the common practice to deploy the backend? npm install everything on the server again? Is webpack a solution? Does it work with nestjs? Any guide on this? I have been googled for a while but can't really find anything about deployment about nestjs. Thanks.

[weeco]

@hkjeffchan My approach is to commit only the typescript files. For each deploy I do a git pull, run npm install and then I compile my typescript on the server too.

I don't think the compiled javascript should be part of your github repositories. On the other hand there are probably better approaches than mine (maybe using Docker?) but I hope this helps you for the first.

I agree I'd love to read how Kamil would handle his deployments though.

[SebastianPodgajny]

@hkjeffchan @weeco

I have managed to bundle very basic nest app with also very basic webpack (v4.3) config

const path = require('path');


module.exports = {
  mode: 'production',
  target: 'node',
  node: {
    __dirname: false,
    __filename: false,
  },
  entry: './dist/main.js',
  output: {
    path: path.resolve(__dirname, 'webpack-dist'),
    filename: 'server.bundle.js'
  },
};

You need to define two new scripts in package.json

"webpack": "webpack",
"start:webpack": "node webpack-dist/server.bundle.js"

and then run

npm run prestart:prod
npm run webpack
npm run start:webpack

When it comes to deploy the best way is to make multistage Docker image and push it to yours private registry

example Dockerfile

FROM mhart/alpine-node:9.8.0 as setup
WORKDIR /src

COPY . /src
RUN apk add --no-cache git
RUN npm install && \
    npm run prestart:prod && \
    npm run webpack
RUN mkdir /temp_build && \
    mv webpack-dist client /temp_build/ && \
    rm -rf * && \
    mv /temp_build/* /src && \
    rm -rf /temp_build
RUN cd client && \
    npm install && \
    node_modules/.bin/ng build --aot --prod --build-optimizer
RUN mv client/dist temp_client && \
    rm -rf client/* && \
    mv temp_client client/dist


FROM mhart/alpine-node:base-9.8 as dist
WORKDIR /src
ENTRYPOINT ["node", "webpack-dist/server.bundle.js"]

COPY --from=setup /src /src

second stage purpose is to squash layers, which in my case results in ~70mb image (base image mhart/alpine-node:base-9.8 - 44mb)

[mattwelke]

Because it's a server side app, there's no need to Webpack, minify, uglify, etc. You can see an example of running a Node app made in TypeScript in Microsoft's starter repo. The tsconfig.json file is set up for CommonJS modules (since Node is able to work with them) and the TypeScript compiler is used to compile the source code from TypeScript to whichever level of JavaScript will work for your version of Node, most likely ES6 or ES7 these days. Your server installs the NPM modules and then runs this JavaScript code.

If using Docker, the installed NPM modules and the compiled JavaScript code are baked into the image.

[weeco]

@welkie We need webpack for the module resolution in nest. There is also a starter for nestjs ( https://github.com/nestjs/typescript-starter ), however it uses tsconfig-paths for the absolute module paths. Webpack does the module resolution as well. The TypeScript compiler itself is not capable of resolving these paths for modules.

[mattwelke]

Ah I see. I only started reading the documentation so far, so I didn't realize there were exceptions that made this different from other Node apps.

I'm not sure what you mean by module resolution. To clarify, are you talking about ES6 modules, or a Nest specific term?

EDIT:

I looked at the TypeScript starter and it actually has no mention of Webpack anywhere. It seems to use a build process identical to the TypeScript-Node-Starter, where individual TypeScript source code files are compiled by the TypeScript compiler into individual JavaScript files, which are then placed in a dist directory, to be run by the process on the server.

Can you clarify how Webpack fits into this?

[weeco]

I just took a look at it again and I noticed that it doesn't use the absolute import paths (anymore?). As I said in my previous comment the NestJS TypeScript starter uses ts-config to solve the issue with absolute import paths (if one wants to use them):

"start": "ts-node -r tsconfig-paths/register src/main.ts",

If you provide a baseUrl in your tsconfig and use webpack or ts-paths you can then import modules like this:
import { PlayersModule } from 'players/players.module'; I set the baseUrl to './src' and therefore it would search for the module at src/players/players.module. Especially for larger projects that can be very helpful (e. g. for refactorings, shorter imports). The TypeScript compiler doesn't touch the import paths during transpilation and Ryan Cavanagh from the TypeScript team said they don't intend to change this because of the given complexity.

[patrickhousley]

I do not recommend using any bundler for Node applications like a NestJS server for two reason.

1. Bundling of modules that have OS bindings, can cause the server to not work in other environments. This can be gotten around with Webpacks externals but still, best just not to do it.
2. TypeORM allows pulling in migrations and entities via glob pattern. If you use that and bundle the app, the glob patterns will no longer match any files and your entities will not be pulled in and your migrations will not execute. Again, this can be gotten around by importing individual entities and migrations but that get tedious on large apps.

Personally, I just let my IDE manage the imports,I use relative paths, and compile with tsc. I understand this can also turn very tedious very fast even for medium sized apps. For that, I would fall back on @weeco's suggestion. Below are some links that might help.

https://github.com/Microsoft/TypeScript/issues/15479
https://github.com/dividab/tsconfig-paths
https://www.npmjs.com/package/tspath

[patrickhousley]

@hkjeffchan Can this issue be closed? If you have further questions, please post them.

[hkjeffchan]

Thanks for all the feedback and time for me to test and study! Close this issue right now.

[CanKattwinkel]

@patrickhousley consider the following example:

I have a repository at Gitlab and my CI is running there, it creates a distribution on each push to the master and deploys it on my web server. When I'm not using a bundler like Webpack I'll run into multiple issues. E.g. node_modules - I'm required to have another npm install on the production server (or copy the whole node_modules folder via network..). Or someone requires the package.json file. What runs local, and in the CI will break on the production server since the package.json is not part of the dist. The project is no longer self contained with view to distribution. The dependency can only be recognized from within the code.

So not using Webpack/Bundler is fine when you build your project on the server you are going to use it. If you have a build pipeline setup on a different device/location it is a problem. Maybe the TypeORM's solution with using globs isn't ideal?

I do not recommend using any bundler for Node applications like a NestJS server for two reason.

Bundling of modules that have OS bindings, can cause the server to not work in other environments. This can be gotten around with Webpacks externals but still, best just not to do it.
TypeORM allows pulling in migrations and entities via glob pattern. If you use that and bundle the app, the glob patterns will no longer match any files and your entities will not be pulled in and your migrations will not execute. Again, this can be gotten around by importing individual entities and migrations but that get tedious on large apps.
Personally, I just let my IDE manage the imports,I use relative paths, and compile with tsc. I understand this can also turn very tedious very fast even for medium sized apps. For that, I would fall back on @weeco's suggestion. Below are some links that might help.

Microsoft/TypeScript#15479
https://github.com/dividab/tsconfig-paths
https://www.npmjs.com/package/tspath

[yoava]

@patrickhousley I disagree with your statement of "I do not recommend using any bundler".

For some projects I love using webpack for the extensive loaders & plugin support: using loaders I can require yaml, toml and tons of other files directly as JS objects, supporting lots of features & DSLs . Using plugins I can easily pack my app for different environments etc.

The issues you mention are easily solved by not packing anything under node_modules (when I do, I usually pack only project's code) or by adding other needed rules (e.g. setting externals in webpack).

Not being able to use any bundler will eventually be a pain.

[patrickhousley]

@CanKattwinkel @yoava valid points. Let's get back to the original issue. @hkjeffchan below are some boilerplate projects that make a point of using Angular and NestJS together in the same repo. I hope these help.

https://github.com/bojidaryovchev/nest-angular
https://github.com/kamilmysliwiec/universal-nest
https://github.com/patrickhousley/nest-angular-universal

One thing you could also look at doing is using something like Yarn workspaces or Lerna and setup your project as a mono-repo. This way, your dependencies are hoisted but each project can be built, tested, linted, etc separately.

[patrickhousley]

Not a very popular npm module but udk I believe is kinda what your are looking for.

[cloakedch]

Has one of you experienced the following error when running a webpack'ed bundle?

[Nest] 23911 - 9/26/2018, 4:41:17 PM [PackageLoader] The "class-transformer" package is missing. Please, make sure to install this library ($ npm install class-transformer) to take advantage of ValidationPipe.

Of course, class-transformer is present in package.json and is installed under node_modules (It is not missing on the disk).
The application can also be run with npm run start or npm run start:prod, so it's really just webpack that is ignoring class-transformer for some reason.

My webpack.conf.js file:

const path = require('path');

module.exports = {
  entry: './dist/main.js',
  mode: 'production',
  target: 'node',
  node: {
    __dirname: false,
    __filename: false,
  },
  output: {
    path: path.resolve(__dirname, 'webpack-dist'),
    filename: 'server.bundle.js',
  },
};

Webpack in package.json:

    "webpack": "^4.20.2",
    "webpack-cli": "^3.1.1",
    "webpack-node-externals": "^1.7.2"

My system:

[System Information]
OS Version     : Linux 4.9
NodeJS Version : v10.11.0
NPM Version    : 6.4.1
[Nest Information]
mongoose version : 5.2.2
common version   : 5.3.9
core version     : 5.3.10

Any ideas?

[coenmooij]

@cloakedch We have the exact same problem but with class-validator. Did you manage to get it to work?

[cloakedch]

@cloakedch We have the exact same problem but with class-validator. Did you manage to get it to work?

Not really. I decided then to at least bundle my app code and exclude all modules from node_modules using this:
https://www.npmjs.com/package/webpack-node-externals

This will require one to issue an npm install though, unfortunately, so it's still not a good solution and doesn't fit my needs (I would like to be able to simply run node server.bundle.js without installing all modules again on the production system).

It seems like the nest developer(s) have different developing goals at the moment, so we are left alone at the moment with this bug...

[marcus-sa]

You don't need a bundler or webpack to use module resolution.
There's project references in TS 3.

[loki29pl]

I have exactly the same problem.

class-validator is loaded by loadPkg function in validation.pipe.js:

classValidator = loadPkg('class-validator');

That's why webpack cant resolve class-validator. Does anyone have a good solution?

[lock[bot]]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.