Nx: NEXT JS: Scss import issue and deployment strategy concerns

[asherccohen]

• [x ] I am running the latest version
• [x ] I checked the documentation (nx.dev) and found no answer
• [x ] I checked to make sure that this issue has not already been filed
• [x ] I'm reporting the issue to the correct repository (not related to React, Angular or any dependency)

Context

I'm creating this issue following @jaysoo suggestion.
Next 9.1.5 has css built-in, so importing css files can be done out-of-the-box. Just add an import to _app.js:

import 'myCss.css';

A normal next app (9.1.5) needs withSass to compile and import scss files though.

Strangely enough, the same project ported to nx workspace (same exact configuration, package.json, dependencies, etc) doesn't need withSass. Running 'nx serve myApp' or 'nx build myApp' works like a charm.

This ties in with deployment, I'll explain why below.

Expected Behavior

'nx serve myNextJsApp' is consistent with running 'next build'. Either way (with or without withSass plugin) we want to be able to build the application, from a nx workspace and out of it.

PLUS: Have a structured strategy to deploy Next Js applications on hosting platforms like Heroku/Now.

Current Behavior

'nx serve myNextJsApp' is not consistent with running 'next build'.
TLDR: nx is correct, next is not. I am opening this issue also on next repo for them to look into.

Steps to Reproduce

A bit of introduction:

WHY:
I want to deploy a next js application from my nx workspace (let's leave CI/CD on a side for now, but the long term plan would be to manage deployment with CI/CD and not manually).

As far as I was able to understand the common nx strategy to deploy an app is to build it and deploy artifacts.

When I run 'nx build myApp' my .next folder is copied to dist/apps/myNextJsApp which is already a problem since the minimal structure for a Next js app to be deployed is:

root

• .next
• public
• package.json
• node_modules

Anyway, let's say I move the folders manually into this structure.

This works on a hosting platform that doesn't have a build step. If the hosting platform has an automatic build step this fails, since there is no project to build (please clarify if I'm missing something here).
Therefore we need to skip 'nx build' and deploy the project folder instead (nxRoot/apps/myNextJsApp) and leave it to the hosting platform to build.

Please provide detailed steps for reproducing the issue.

1. generate a next app with nx workspace.
2. import scss in _app. The scss I am trying to import is a main.scss file with many partial @imports, also some properties have url() imports referencing static assets. Just clarifying this as I haven't tried with a simple scss file).
3. run nx serve myNextJsAp: it should work without problems.
4. close, navigate to nxRoot/apps/myNextJsApp.

Now let's prepare the project for deployment:

1. copy package.json from nxRoot to nxRoot/apps/myNextJsApp and replace scripts to:

"scripts": { "dev": "next", "start": "next start", "build": "next build", }

1. OPTIONAL: remove dependencies that are not strictly related to the app.

Lastlylet's mimic what a hosting platform would do after pushing this project (I use git subtree for heroku or navigate to the folder and use now cli for zeit now):

1. run 'yarn install' or alternative 'npm install'
2. run 'yarn start': see it fail since withSass plugin is needed to import scss files.

@jdpearce Please reply with detailed requirements and I can add them.

[asherccohen]

UPDATE: Just a brief add on. My test assumes no shared libs/shared components because I seriously wouldn't know how to let the hosting platform build without knowledge of the full nx workspace.
This is a different matter, I guess we can live without shared libs for now, although it kinda denies the purpose of a monorepo.

[asherccohen]

UPDATE 2:
Ok after fiddling with this a lot I noticed I can deploy the built folder (build artifacts) to Heroku, by removing the build command in my package.json the hosting platform jumps the build step and starts the app.

BOOM.

So my suggestion for the team is to modify the folder structure when running 'nx build myNextJsApp' command to:

nxRoot/
dist/
apps/
myNextJsapp/

• .next
• public
• package.json

Like this we can git subtree the build folder and we're back running, no need to fix the scss problem (but interesting to know why it happens anyway).

Problem still persists on now, as I couldn't find a way to jump build step.

[asherccohen]

UPDATE 3:
@jdpearce explained to me why withSass plugin work out of the box on nx while it doesn't in a plain next js app.
Basically the nx team wraps next.config.js in an enhancer with predefined plugins already installed. One of these is withSass.
Let's discuss here if devs prefer having full control over next.config.js, hence removing the enhancer, or not. Thanks.

[krizic]

@asherccohen hey man I'm having an issue with similar problem, we still haven't found the way to create and run deployment with scss and custom server.js file. https://github.com/nrwl/nx/issues/2478 more described here with example repo.

I would highly appreciate if you give it a look and let me know if you see a solution. We are kinda blocked from any deployment before we figure out how to do this.

[asherccohen]

Hey @krizic!Sorry to hear that you're stuck!
I had a quick look at your repo and from what I can see you're using all defaults from nx which is good.
I must be honest I never tried serving an app with a custom server like that (customServerPath config), is there a particular reason why you need to do that?
'nx serve myNextApp --prod' is what you're looking for while developing locally.
Also using the nx dev server in production or on any hosting platform is highly not recommended.
I've had a long conversation with the nx team and bottom line is that it's not designed for it.
This is the reason of my thread here actually since nx provides awesome tools for local development but it's opinionated on deployment.
The rule of thumb if to build your project (even on CI) and then deploy artifacts.
If you're saying that your issues appears only in production I would do the following:

• remove the custom server and config.
• change the output dir for your nx build step to .next (this way you get the native structure of a next app in your dist/apps/myNextApp)
• nx build myNextApp --prod
• your artifacts (including shared libs and components) are now in your dist folder.
• create a deployment specific package.json and place it in dist/apps/myNextApp with all dependencies for your project (except shared libraries from nx and other nx related deps)
• add nextJs specific scripts to start your app from its docs (I only experimented with heroku and you need to make sure it doesn't try to build again on push, avoid a build script to do so, can't tell for other hosting platforms)
• if you have static assets you need to copy also your public folder to dist/apps/myNextApp. As a reminder public folder now replaces static folder for next 9. ( I have opened a separate issue foe this matter as this step should be done automatically by nx build)
• git subtree the dist/apps/myNextApp folder as if it was a standalone project
• hosting platform should have node capabilities (unless your building a static project, but that's a different process I'm afraid) so it can start your app with nextJs server.

It's tedious and I'm sorry if it looks like I went off topic but do feel this is related.

Another option is to try with a custom next.config.js file, just in case the plugins are acting up again. You can find examples on next js docs.

[krizic]

@asherccohen Hey thank you for your help, however custom server.js is required in our case due to authentication checks. I believe I get the very same result in case I just do a prod build and try to serve it from .next folder. The page still requests an unexisting module "index.scss"... even when built. I double-checked and indeed in file there is a reference for it.

Thank you for the rest of the recommendations, those really help. I will apply them as suggested.

However it seems we are still blocked from deployment... :/

[krizic]

@asherccohen I just tryed what you recommended

I prod built project and moved it into dist/apps/mpyl-ssr/.next
I copied public folder and added also moved everything from dist static folder into it.
I removed my server.js and next.config
I tryed to run solution with "start": "next dist/apps/mpyl-ssr/.next", from the root of the nx workspace.

and I get:

[ wait ]  starting the development server ...
[ info ]  waiting on http://localhost:3000 ...
Error: > Couldn't find a `pages` directory. Please create one under the project root
    at findPagesDir (/Users/vkrizic/src/***/node_modules/next/dist/lib/find-pages-dir.js:3:170)
    at new DevServer (/Users/vkrizic/src/***/node_modules/next/dist/server/next-dev-server.js:1:3638)
    at createServer (/Users/vkrizic/src/***/node_modules/next/dist/server/next.js:2:105)
    at start (/Users/vkrizic/src/***/node_modules/next/dist/server/lib/start-server.js:1:323)
    at nextDev (/Users/vkrizic/src/***/node_modules/next/dist/cli/next-dev.js:22:279)
    at /Users/vkrizic/src/***/node_modules/next/dist/bin/next:29:346

I noticed that the pages folder is actually: dist/apps/mpyl-ssr/.next/public/jRsTYnQyPbTCQ0ZlZd89Y

Do you have any advice?

[asherccohen]

@asherccohen I just tryed what you recommended

I prod built project and moved it into dist/apps/mpyl-ssr/.next
I copied public folder and added also moved everything from dist static folder into it.
I removed my server.js and next.config
I tryed to run solution with "start": "next dist/apps/mpyl-ssr/.next", from the root of the nx workspace.

and I get:

[ wait ]  starting the development server ...
[ info ]  waiting on http://localhost:3000 ...
Error: > Couldn't find a `pages` directory. Please create one under the project root
    at findPagesDir (/Users/vkrizic/src/***/node_modules/next/dist/lib/find-pages-dir.js:3:170)
    at new DevServer (/Users/vkrizic/src/***/node_modules/next/dist/server/next-dev-server.js:1:3638)
    at createServer (/Users/vkrizic/src/***/node_modules/next/dist/server/next.js:2:105)
    at start (/Users/vkrizic/src/***/node_modules/next/dist/server/lib/start-server.js:1:323)
    at nextDev (/Users/vkrizic/src/***/node_modules/next/dist/cli/next-dev.js:22:279)
    at /Users/vkrizic/src/***/node_modules/next/dist/bin/next:29:346

I noticed that the pages folder is actually: dist/apps/mpyl-ssr/.next/public/jRsTYnQyPbTCQ0ZlZd89Y

Do you have any advice?

This wouldn't work, since the root of the workspace is not aware of the dist/apps/myNextApp structure. You would have to configure the paths properly and I suggest against it.
What you should do is:

• no need to copy .next from the project to the dist folder. Just go in the workspace.json and change the outpur dir for your next js app. Change it to .next, this way 'nx build myNextApp' will put artifacts in it.
• move the public folder from the project to the dist folder. (NOT next/plublic BUT the public folder that sits at the same level of next.config.js)
• add a package.json as described (with scripts)
• cd to dist/apps/myNextApp and run yarn install
• once node_modules are installed locally in the dist folder you can run 'yarn start'.

Think of dist/apps/myNextApp as a separate project out of the monorepo, like your CI would do when you deploy. You need to install packages in that folder and then run the app.
Is it a bit more clear now?

[asherccohen]

@asherccohen Hey thank you for your help, however custom server.js is required in our case due to authentication checks. I believe I get the very same result in case I just do a prod build and try to serve it from .next folder. The page still requests an unexisting module "index.scss"... even when built. I double-checked and indeed in file there is a reference for it.

Thank you for the rest of the recommendations, those really help. I will apply them as suggested.

However it seems we are still blocked from deployment... :/

You're right I didn't think of that scenario, worth investigating next js /api folder then, since version 9 supports building apis next to your app. If that doesn't suit your case I'm of the _opinion that you still don't need to run the server.js file directly, if you deploy as I recommend then 'next start' should be able to run the server for you._

EDIT:
Sorry, I made a mistake here. It is suggested to use /api going forward but you're right saying that you can run your custom server with a package.json script like this:
"start": "node server.js"
Then with a 'npm run start' your requests will go through your custom server.

[jaysoo]

Let's discuss here if devs prefer having full control over next.config.js, hence removing the enhancer, or not. Thanks.

I believe @jdpearce is leaning towards allowing next.config.js to be controlled by the user, which would make sense if it needs to be extended upon. I think Nx should still provide some sane defaults so things work mostly out of the box.

[jaysoo]

We've decided to support next.config.js instead of internalizing everything in Nx. Will work on something for the next release.

[jaysoo]

There's a PR now to expose the next.config.js file by default (and automatically installs and uses the config enhancer matching the chosen style).

This should help a lot with folks trying to customize the config further.

Next up we need to add support for copying assets from the app to dist folder. This should be similar to other builders (such as @nrwl/web:build) that support the assets option to copy. files.

[jaysoo]

Update: I have a draft PR that structures the dist output to match a normal Next.js project.

https://github.com/nrwl/nx/pull/2792

I've tested it with Heroku and seems to be working well. Now I just have to make sure we have a builder to serve it (i.e. repurpose the current dev-server builder).

[jaysoo]

Update 2: The PR is ready for review. I tested it with Heroku, using their Docker container support.

[dabbeg]

Next up we need to add support for copying assets from the app to dist folder. This should be
similar to other builders (such as @nrwl/web:build) that support the assets option to copy. files.

@jaysoo You mention here implementing copying of assets but as far as I can tell it is not a part of your PR. I would really like having this feature in the next builder.

[github-actions[bot]]

This issue has been automatically marked as stale because it hasn't had any recent activity. It will be closed in 14 days if no further activity occurs.
If we missed this issue please reply to keep it active.
Thanks for being a part of the Nx community! 🙏