Next.js: Custom server logic / routing

[rauchg]

next -s server.js

or in package.json:

{
  "next": {
    "serverHandler": "./server.js"
  }
}

Let's say you want to override the default mapping between filesystem and route, you could set up something like this:

const { renderComponent } = require('next/server')
module.exports = ({ path, req, res }) => {
  if ('/a' === path) {
    return renderComponent(req, './b')
  }
}

The function would receive a context object with the following properties (as far as I have thought about it):

• path the intended path of the request. This is so that if we request /a.json or /a, the path is still /a, which simplifies the logic for the programmer
• req the original request from node
• res the original response from node

The return value can be a Promise. If undefined is returned, it performs the default behavior.
This file provides an opportunity to short-circuit (like send a 301), add global security / authentication requirements, etc.

[1] We might or might not want to transpile this file. Since it's "server-only", I think it's ok to assume it just supports whatever the version of Node.js you're running.

[nodegin]

Can't get this this to working, I have tried to create a server.js

const { renderComponent } = require('next/server')
module.exports = ({ path, req, res }) => {
  console.log(path)
  if ('/a' === path) {
    return renderComponent(req, './pages/index')
  }
}

but still showing 404 with no logs printed on console

[dotcypress]

@nodegin this feature isn't implemented yet. 😢

@rauchg do you wanna PR for that?

[rauchg]

This should be fairly easy to implement. Wanted to hear comments about whether the proposed API makes sense to you all.

[malixsys]

Might be easier to have a convention on say /api/* matching .js files in a server directory...
Maybe also supporting some koa/express type middleware insertion/signature (req, res, next) => ?

[jmfirth]

Without realizing this issue existed I have done some work toward this already. It allows for custom endpoints placed in an /api folder. Perhaps the modifications might interest someone here: https://github.com/jmfirth/next.js/commit/c42b56217dc29a5a4a50b5bdac6ba2b771e37cec

Outline of changes:

• Add new webpack entry points
• Modify applicable webpack loader include/exclude conditions
• Add custom render method
• Add a route entry

An example endpoint /api/hello.js:

export default function ({ req, res, params }, { dir, dev }) {
  return { message: 'Hello world!' }
}

And response from /api/hello:

{"success":true,"body":{"message":"Hello world!"}}

[seanyesmunt]

@jmfirth That looks great and is exactly what I am looking for. I can also see it being very useful as I have a really small app with only a few api calls. It would be awesome not to have to create a separate server project for my app.

What are the dir and dev variables?

[jmfirth]

dev is presumably a development mode flag

dir appears to be an injectable root module path that defaults to process.cwd()

A more general answer is that it follows the same general convention that the render method follows. Both seem like they could be potentially useful for someone developing endpoints.

[nodegin]

this should be merged into master

[davibe]

After reading the home page of the project and this issue some things are not clear to me. Please, consider that i only have a very superficial understanding of the project but maybe my doubts can help you clarify some things about the project itself even in the home page.

1) code may need to be able to do something when both entering on a route and leaving a route

2) If you perform some task (i.e. data fetching) it needs to be clear wether re-entering the same route causes a redraw or not

3) sub-routes like product/:id/comments/:cid/author?expanded=true means that a child component may need to access "parent" parts of the url, the data fetched by parent components, url parameters.

4) with the lack of client-side state and routing, the only thing left to be universal is just the views code (?). I may be missing something but that sounds like too little to call it universal. What's the advantage beside components and their reusability?

[jaredpalmer]

Would you allow for renderJSON too?

[rauchg]

I think it'd be really smart to expose a micro type contract: you can resolve the promise to an object and it gets rendered as JSON automatically :)

[rauchg]

Also, if you want to _parse_ incoming JSON, you can just use micro tools!

import { json } from micro
export default async function ({ req, res }) {
  if (path === '/woot') {
    const body = await json(req)
  }
}

I like this a lot as it preserves minimalism and great performance while allowing for unlimited power!

[rauchg]

@davibe

1) You can do this with lifecycle hooks. didMount / unmount
2) This is also up to react and componentWillReceiveProps, shouldUpdate, etc
3) You parse the url, you decide what component gets rendered by turning the URL into query data. The URL can be "polymorphic", ie: it can point to any component.
4) We support both client side state and routing. Redux example: https://github.com/zeit/next.js/wiki/Redux-example. I don't see how you can be more universal than this.

[dotcypress]

Will be nice to have method for serving static files.

import { sendFile } from 'next/server'
export default async function ({ path, req, res }) {
  if (path === '/foo/bar.png') {
    await sendFile(res, './build/bar.png')
  }
}

[jaredpalmer]

@dotcypress you can make a folder called static and just drop stuff there. it maps to /static/thing.png

[jmfirth]

@rauchg I spent some time this morning refactoring the example I posted above and am centering on a similar approach with differences:

• I want all the custom code transpiled, so I expanded the entry point glob defined in webpack.js to include everything but node_modules and static folders as potential entrypoints:

const entries = await glob('!(node_modules|static)/**/*.js', { cwd: dir })

And modified all the JS loader constraints to simply: exclude: /node_modules/ which allows for webpack to transpile all custom code. At least for my purposes this makes a lot of sense.

• I saw your points about registering the plugin explicitly in package.json or at the CLI. For this example I went with anything in a /plugins folder instead. One of the strengths of this platform, in my view, is the directiory structure assumptions in conjunction with many dynamic entry points. It seems like this strength should prevail.
• I want the plugin to pick up at the route definition point, but I don't want them to have access to this.router directly, so I added a helper method to server/index.js called addRoute that is injected into the plugin:

  addRoute (path, fn, method = 'GET') {
    this.router.add(method.toUpperCase(), path, fn)
  }

• The plugin's hook becomes a route definition and each script is a simple default export of that definition. I also think this is probably the right level- it's hard to predict what the plugin might be so having access to the res makes a lot of sense.

export default function (addRoute) {
  addRoute('/api/:path+', async (req, res, params) => {
    // do something and manipulate `res`
  }, 'GET')
}

This approach still has a way to go but I like how it's all coming together in my test app.

[jmfirth]

A concrete example of changes: https://github.com/jmfirth/next.js/commit/d63bf5349c09c28a6a3f0e28e2abd7ceae0090de

And example plugin that enables the same sort of /api routes - /plugins/api.js: https://gist.github.com/jmfirth/a202d0dc9c52c64be6e8523552e0fc4a

[nkzawa]

I wonder if we can extract some parts of next as a library, instead of supporting custom server and api things.
Then ideally, you can create your server using http or express etc.

import next from 'next-core'
import http from 'http'
const render = next({ /* settings */ })

http.createServer((req, res) => {
  if (req.url === '/a') {
     render(req, '/b')
     .then((html) => {
       res.setHeader('Content-Type', 'text/html')
       res.end(html)
     })
  }
})

[rauchg]

@nkzawa I was thinking that taking complete control of the process would be really cool! Specifically, so that you can attach other things to the http server, handle upgrade ws event, etc

I still think that next build will be highly desirable, so maybe instead that module should perform export defaults of the created server?

[rauchg]

Alternatively, this is one of the examples where "ejection" could be really cool to implement. Let's say you don't want ./pages auto-registration at all, or you want a completely different path, you can "eject" the server part and customize it.

[nkzawa]

What if you can write a server using any modules, but can't use next start or next.

// ./some/path/my-server.js
const { renderComponent, serveStatic } = require('next/server')
const http = require('http')

http.createServer((req, res) => {
  // ...
}).listen(3000, () => {
  console.log('Ready!')
})

and run:

$ node some/path/my-server.js

[rauchg]

I love that idea provided that renderComponent works well with the pre-built cache of next build. We'll also want to make sure the entry points are customizable, in case people really want to completely change the fs structure.

[malixsys]

Using a micro instance (or Koa2) would be so useful! Don't reinvent the wheel :)
What do we need to get this merged?

[reel]

One use case: using next on top of FeathersJS... Been doing some basic ssr with FeatherJS and the output of CRA build but, using next would be great (as a hook for express app.get('*', nextjs(req, res, next));

[thisguychris]

This would also provide some escape hatch if I want my code to be 'server only' and not universal.

@jmfirth correct me if I'm wrong, but your PR actually makes this escape hatch universal rather than server only?

[mbilokonsky]

I'm probably in the minority here but I've come around to the perspective that I wouldn't want next to handle all this stuff directly. I like the idea of wrapping the whole thing in a middleware tier, and routing to next as appropriate. That way you're not using next as a one-size-fits-all solution, its role gets restricted to rendering a view as it does now.

Curious for thoughts around this, though. Do most people find it simpler just to add this functionality to next, rather than abstracting it out to a higher level?

[mbilokonsky]

Specifically, rather than adding server support to next, I feel like there's one tool missing from the zeit stack - an orchestration layer that essentially just exposes middleware routing. It becomes your server, and each endpoint of your server gets delegated to a standalone micro project.

So let's call this project orchestrator or something. You'd npm install orchestrator, and it would give you some CLI commands.

It could step you through some basic questions, do you want a view layer, do you want a service layer, do you want secrets and environment variables, etc. It stores all of this to an orchestra config file.

If you opted to include a view layer, it could create a subproject that uses next to stub that out. (or, down the line, next could be one of several offerings - could do angular or elm or clojurescript stubs here, too, this could be pluggable).

If you opted to include an API layer, it could create an 'api' folder with one default micro instance that just sings at /api/ping and returns pong, with some stubbed out tests for it.

You get a routes file that allows you to parse request URLs as @rauchg mentions above and maps them to specific client pages or service endpoints.

The whole thing has git support to treat it as a single project, but each service and each next project gets its own deployment logic to get pushed out to autoscaling now instances. So you have the feeling of working on a single application, but under the hood it's all pieces and parts that know how to work together.

You could have npm run dev to run all of your services locally on different ports (and your middleware tier can just be aware of this), or you could have npm run deploy:dev which deploys your services out to now instances with your dev envars set, or npm run deploy:prod which deploys them with your prod envars sets and auto-aliases the middleware host to your alias.

I've got a prototype that has started down this road, using now-fleet to handle orchestration and deployment, but there are some things I am still trying to iron out. But I vastly prefer this degree of abstraction, and this orchestrator can be the one-stop-shop turnkey product for stubbing out and deploying huge sites.

And that's why I think it's a mistake to add custom server support to next, just like it would be for micro. Because ultimately your view engine is just a specialized microservice that returns HTML instead of JSON.

/phew rant over but I'd love to hear thoughts about this.

[rauchg]

@mbilokonsky I agree that a lot of this can be handled at the proxy level. But we also have to be realistic in that people will need at least _some_ access to the underlying server. This API tries to expose the most minimal surface for doing that, without introducing or advocating bad practices (like tucking your entire server API into one process).

I'm thinking about writing a document about "best practices" in terms of separation of APIs in the backend for Next to avoid that situation

[eirikurn]

This would be perfect for sites powered by a CMS api. The handler could query the CMS using the route data and render different page (templates) accordingly.

[evantahler]

For custom routes, why abandon the core the file system is the API rule? I think that is one of the cooler parts of this framework! The biggest problem I'm having is regarding RESTful routing, IE: I've got ./pages/user/view.js which I would like to render for /user/:id. I've got no problem reading the URL... it's just telling Next which partial to render.

To that end, I suggest using express/rails style route conventions _in_ the file names. So in my example, I would literally name my file ./pages/user/:view.js, and it would work. The name after the : is actually meaningless, here, but it would be cool to inject that into the default exported component as a prop too..., or as part of the url prop that is already being passed in.

[evantahler]

Solving my own problem (https://github.com/zeit/next.js/issues/25#issuecomment-258660345) with https://github.com/zeit/next.js/pull/223

[eirikurn]

I like making the "file system as an api" more powerful, but I also consider that the "default resolver", allowing more powerful url handling when needed.

Of course this project should not be everything for everyone. But this extension point would open up a lot of cool possibilities. A CMS backed React site being just one of which.

Speaking of which, it would be nice if renderComponent supported additional context for getInitialProps. This could be used to parameterise the page based on the url and external data (saving a potentially redundant fetch).

[tpreusse]

I would stick to just routing and think that rails does it well:
config/routes.rb -> routes.js

routes.js is guaranteed to only run on the node side and is allowed to access fs. But it is sync as it's not expected to change without a deploy – does not depend on something remote.

This would look something like this:

const {route} = require('next/server')
module.exports = [
  route('/about', {to: 'about'}),
  route('/user/:username', {to: 'user'})
];

to can only point to files in pages (which are webpack entry points / chunks).

/user/tpreusse would be _routed_ to pages/user which gets params: {username: 'tpreusse'} in getInitialProps. Additionally getInitialProps would expose path_for and url_for which could be used like this: path_for('user', {params: 'tpreusse'}).

I think in the solution described in the issue description you would still need a way of passing information to the page / component rendered (as @eirikurn also mentioned) and the imperative nature would make it hard to predict routes. My assumption is that with unpredictable routes the code splitting could become hard or sub optimal.

How would renderComponent work? Wouldn't it be more of a renderPage?

[auser]

Bump: Curious if there is any movement on this PR?

[rauchg]

Yes. I'm going to write a ticket specifying the final API we will implement
to request comments before shipping ✌️️

On Sun, Nov 20, 2016 at 9:15 PM Ari [email protected] wrote:

Bump: Curious if there is any movement on this PR?

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/zeit/next.js/issues/25#issuecomment-261817446, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AAAy8YazyRi1KR7WCBmcft5jYfY0KEwaks5rAOKIgaJpZM4KYHwp
.

[rauchg]

Closing this in favor of "Programmatic API": https://github.com/zeit/next.js/issues/291.
Thanks a lot for your feedback!

Keep track of that issue to learn about the progress of its implementation.

Closing this as it's no longer an exploration.

[rdewolff]

Like @rauchg mentioned, having a similar function to the "eject" of create-react-app would be great.

It would comfort users that doubt to use Next.js because of the "closed environment".

[tomsoderlund]

Is @jmfirth's suggestion implemented, or is a custom Express server the way to implement an API (custom backend)?

[timneutkens]

Custom server is the way to go 👍