Hyperapp: SSR

[jorgebucaran]

Similar to what we are doing in #238, I'd like to start a new issue to discuss SSR with HyperApp.

The purpose of this issue is to decide what a SSR app in HyperApp would look like, so let's not get bog down in implementation details. We also need a minimal code example to share ideas. @benjaminj6's example here is a good starting point.

/cc @andyrj @SkaterDad

[andyrj]

simple koa ssr example

I will make branches that show simple-dom and undom as well... just slapped this together real fast by extracting it out of my full starter with hmr and what not...

for that example, simply npm install, npm start and open browser to localhost:3000 for a ssr rendered counter example

[andyrj]

I am not endorsing any specific render method, I think writing our own is best, but for the mean time we can use one of the dom libs...

[naugtur]

I guess the key part will be to get it to rehydrate without adding much to the 1KB :) or do you expect dom diffing would just do it (tm)

Correct me if I'm wrong, but if we don't do something really weird with Components, rehydration should be independent of them.

[andyrj]

I tried slapping undom and simple-dom in that repo on branches but neither of those libraries have document.readyState[0], so when hyperapp runs it dies on 'cannot read propery [0] of undefined', jsdom has readyState so it still works though for now.

@naugtur I agree that rehydration and components should be unaffected by one another

Realistically the code I slapped in https://github.com/hyperapp/hyperapp/pull/255 may not be optimal or minimal but was working on my little sample starter. I'm sure we could put a hook in there that allows us to remove that code from the render function itself and make it optional/mixin or something etc... But after the first render the if statement around the 2 lines that change how render is running will be short circuited at the first condition element !== undefined, so it should have negligible impact unless I screwed up the other conditions/didn't understand the render/patch well enough. It also shouldn't have an impact on people not using SSR because that same conditional is checking the render target element to see if it has children before entering that new logic to "hydrate"

To bring up the one thing I am concerned about from the test cases I wrote in that PR. Is this test: https://github.com/hyperapp/hyperapp/blob/master/test/root.test.js#L29 , accurate/expected behavior?

That test is basically against rehydration since it seems to be some special case where rendering into existing dom has some logic where it skips initial nodes and looks for a matching tag or something?
I didn't look into it further but I expected the render target to be either, explicit, or document.body from the docs. Not explicit, or document.body within the first matching tag. But maybe I'm just not understanding something.

Edit: One more note on that PR
In that little example hydrate https://github.com/hyperapp/hyperapp/pull/255/files#diff-bd9c9dcd314f2d7df52935b3a6a4d504R82 , I am stopping short of actually creating the full vdom with attributes, and this appears to be "ok", but I am not sure there aren't edge cases where this may cause a problem with certain browsers that don't like when you try to add an already existing attribute to a dom element.

[naugtur]

Where do I look to see how rehydrating a click handler works?

[andyrj]

@naugtur that is what I am talking about in the Edit: One more note on that PR above your question.

Well it is basically taking advantage of the standard patch process, since I don't define attributes/event handlers in my, data: {}, of my little hydration, the first render process sees my hydrated nodes and gets upgraded to a patch and the standard patch process will take care of the event handler and all other attributes/properties in your view.

I had been mucking around with where was the least amount of changes I could make to try to get that working yesterday for quite a while before I decided to submit that PR to discuss.

General idea is to just have a matching Node structure for the tags to avoid the patch process re-creating them and doubling up the dom nodes, and then when the patch happens it will think oh hey look at these elements that are missing all their attributes I should add those, and it just keeps working like it normally does.

[naugtur]

ok, so the items that have event handlers get rerendered. React probably does the same. I wonder if it'd be possible to delegate all events and only have one listener to clicks on app root. That's probably too much for a microframework

[andyrj]

@naugtur it shouldn't re-render but it will behave as if the last render didn't have attributes/events and for some reason this current render does so it will go through the steps to add those attributes/handlers like it normally does. I could have written more code to completely make a vdom from the dom and added the attributes and handlers myself but we already have code to do that so I thought why not reuse it. That would be a lot more code with little benefit when I could simply use what was already there in a clever fashion so long as it works ;)

[naugtur]

sure. makes sense

[andyrj]

@jbucaran for reference on my choosing the term hydrate, the logic https://github.com/hyperapp/hyperapp/pull/255 is a very simple version of what at least starts here in the react fiber project... https://github.com/facebook/react/blob/a37012a6b5fb5a1c0c19c962737189aeaebe3684/src/renderers/shared/fiber/ReactFiberBeginWork.js#L375

I know there was some discussion about that last night, didn't have this link handy at the time.

[andyrj]

ok, sorry for that last comment, now it works with undom: https://github.com/andyrj/hyperapp-starter/tree/undom was able to mock documentReadyState instead of changing anything in hyperapp

Edit: adding list of functions we could scrounge from undom to make an even more minimal dom implementation for hyperapp...

if developit and everyone is ok with it I thought we could also just pull out the functions we use from there currently... or we could fork that lib and make a hyperdom ;) lol

Node.appendChild,
Node.insertBefore,
Node.replaceChild,
Node.removeChild,
Element.children,
Element.setAttribute,
Element.removeAttribute,
Element.addEventListener, // not really needed on server side
Document,
createElement,
createElementNS,
createTextNode,
createDocument

I believe that's all I came up with matching the undom lib functions from my grepping...

Edit: after testing smallest undom can be made is 897B by removing all code we don't currently use
hyperapp-server was 541B, so for +356B and we have mixins and what not, maybe there is some other stuff I missed in undom that we can make smaller too ?

https://github.com/andyrj/undom

[jorgebucaran]

This discussion is going to stagnate (I won't be able to _action_ anything) if we don't have a minimal code example (it doesn't have to work at all) to share and discuss ideas.

EDIT: Grammar.

[nitin42]

I was exploring this stuff and have made an example (imaginary). Here it is -

const DOMStream = require("hyperapp-dom-stream");
const express = require('express');
const app = express();

app.get('/', (req, res) => {
   DOMStream.renderToStaticMarkup(<SomeComponent prop={value}/>).pipe(res);
});

Here we are rendering the static pages. We could use renderToString method to render an element to a readable stream on the server. Or we could combine renderToString and renderToStaticMarkup to serve a page by generating the HTML markup using renderToStaticMarkup and then embedding the output of App using renderToString.

Why streams?

Because streams helps in doing async rendering by sending the first byte and then perform renderToString instead of sync rendering where the server has to wait until the entire HTML is created and there is more CPU usage.

But if we are making any changes regarding the api (introducing components) then we may have to wait on this.

[andyrj]

Uh, the links to the repos above all have working examples as minimal as I am going to want to do. I really think the best approach for basic non streaming ssr is to just adapt undom, we can fork and name it hyperapp-dom, or whatever you like. Then normal calls to app should just work server side, like the link at the top of my previous comment.

[nitin42]

How about this https://github.com/airbnb/hypernova ?

[jorgebucaran]

@andyrj Can you summarize what examples you suggested (and links)? Also see this comment.

[SkaterDad]

Going through the trouble of replicating the DOM in Node seems like it would cause a major perf hit.
hyperapp's node structure is simple enough that toString() functions are trivial.

@jbucaran You seem to just want pseudo-code to help with the API design?

const express = require('express');
const app = express();

//server version of hyperapp & router, could be nearly the same.
//router on server would inspect the "request" object instead of "location".  Would only need match function?
const { hyperapp, router }= require("hyperapp/server");

let myAppConfig = require('./my-app-config')(router)
// Same app config for client & server, but takes Router mixin as param
// { state: ..., actions: ..., view: ..., mixins: [Router, etc...]}

app.get('*', (req, res) => {
  //Naive approach where you instantiate a new Hyperapp on each request.
  //Probably not ideal, and you would want to cache these based on something like the request URL?

  //Server-side hyperapp would accept an app config just like the client version
  //Either return a stream of HTML strings, or a single one.  Express's res.send can handle either.
  //I think it's important that lifecycle hooks are executed so that mixins and such can be applied.
  //For example, the server-side router needs to run its `match()` function before rendering can start.
  //Also need to ensure that any async actions (Promises) are resolved - like http requests.
  const html = hyperapp({...myAppConfig})

  res.send(html)
});

[andyrj]

import { h, app } from 'hyperapp';
require('undom/register');

function serialize(el) {
  if (el.nodeType===3) return el.textContent;
  var name = String(el.nodeName).toLowerCase(),
    str = '<'+name,
    c, i;
  for (i=0; i<el.attributes.length; i++) {
    str += ' '+el.attributes[i].name+'="'+el.attributes[i].value+'"';
  }
  str += '>';
  for (i=0; i<el.childNodes.length; i++) {
    c = serialize(el.childNodes[i]);
    if (c) str += '\n\t'+c.replace(/\n/g,'\n\t');
  }
  return str + (c?'\n':'') + '</'+name+'>';
}

// needed for hyperapp to render with undom
document.readyState = ["1"];

app({state, view, actions})

// however you want to connect http/express/koa etc...
const appOutput = serialize(document.body);

if we fork undom and make it our own this will become as simple as.

import { h, app } from 'hyperapp';
register('hyperapp-server');

app({state, view, actions, mixins});

const appOutput = document.body.innerHTML;

@SkaterDad
I don't disagree that this will have lower performance (though I doubt by much) than the hyperapp-server/renderToString method. However this approach supports mixins and any other functionality out of the box (and doesn't need to be in core), whereas with toString you will need to either integrate it tightly with hyperapp core or recreate the app(), which I think will be an issue for maintainability down the road.

I was able to strip a few hundred bytes of unused code from the undom library in a fork that I think adding the serialize and document.readyState to would be sufficient for a full featured SSR without requiring tight hyperapp core integration or causing maintenance issues moving forward. https://github.com/andyrj/undom

@nitin42
As far as streaming goes I'm not 100% sold on it being useful or required for hyperapp. I feel like React/Vue etc who have streaming rendering options have that because the components they render may have async operations gathering data etc which it then makes sense for. But with hyperapp I would see any async data fetching normally being done before the call to app({state,view,actions}) when you build the state to pass. This may or may not change depending on components etc but for now I think it best to stay with a synchronous ssr setup and we can re-evaluate streaming render later.

[SkaterDad]

@andyrj You're probably right that undom wouldn't add _too_ much overhead, and would be a quick starting point if it allows hyperapp to run without mods. Does undom handle populating the window.location?

Streaming support isn't simply for async actions, it's for breaking up the stringifying into smaller chunks that can streamed to the browser, similar to how a normal HTML file gets sent. The earlier the first byte, the earlier the browser can start parsing and fetching resources.

1. Start stream by sending <html><head>...</head><body><div id="app">
2. Render hyperapp view to string
3. Close stream by sending </div></body></html>

I think it's a reasonable assumption that many apps will have async data fetching happening on hyperapp init (just like on route entry on client). Asking people to figure out which data to fetch before calling app(...) would be pretty tedious. Since you're proposing we can run our apps without modification on the server by using undom, async stuff should just work?

Perhaps the SSR-related stuff (waiting for a signal to stringify, then firing off the response) could even be implemented as a mixin?

const { app } = require('hyperapp')
const SSR = require('hyperapp/ssr')
const express = require('express');
const server= express();

const myAppConfig = require('./my-app-config')

server.get('*', (req, res) => {
  // faking browser DOM/document/window to allow apps to render with no changes or special version of hyperapp & router?  
  require('undom/register');
  window.location = {url: req.originalUrl, etc...}
  document.readyState = ["1"];

  const template = { start: '<html><head></head><body>', end: '</body></html>' }
  const config = {...myAppConfig}
  config.mixins.push(SSR(template, res.send)) //pass app shell template and send function to the SSR mixin
  app(config) //SSR mixin waits for an event and streams response.
});

[andyrj]

@SkaterDad Yes setting window.location is useful for the routing, we could provide a helper for that, or adjust how undom is initialized to provide it.

Edit: below
I am 100% on board with the above implementation.

[andyrj]

@SkaterDad
https://github.com/hyperapp/server/pull/6 , made pull request with basically what you outlined, need input/help with streaming portion.

[andyrj]

I wanted to attach references to this discussion for possibly not needing to add hydrate() to hyperapp.app but instead serializing the vdom server side into a script tag that can be parsed client side before first render when it exists...

https://github.com/hyperapp/hyperapp/pull/282#pullrequestreview-47919151
originated: https://github.com/hyperapp/server/issues/7

This would allow for hydration to simply become a call to JSON.parse() client side when render target has pre-existing dom children on first render, which would move hydrate(or something like it) to ssr instead of core hyperapp.app.

Edit: correction
I think this is worth discussion but complicates streaming render case were you would have to include this serialized json script tag to bottom of body.

[SkaterDad]

@andyrj That discussion in hyperapp/server#7 was pretty confusing to me. Even if you serialize the vdom tree, you still need to modify hyperapp core to use it, right?

The big drawback to that approach in my mind is that you're sending the serialized vdom and the HTML markup. As you said, more bytes sent to the user, and probably slower performance on server side since you have to generate all of those unique IDs while stringifying.

[andyrj]

@SkaterDad
I agree, but I wanted to see what others thought since it would be less code in core. I was also confused in https://github.com/hyperapp/server/issues/7 about what was gained by the data-id, and kind of thought of this while trying to clarify what it was being proposed for.

[SkaterDad]

@andyrj @jbucaran
I've been doing some experimenting with a server-side app() function, including waiting for async actions on init/loaded events automatically.

I still have some cleanup and API tweaking before I'll push to Github, but want to get a thread going for discussion.

@jbucaran Where do you want to keep the discussions and planning around SSR? In this main repo, or in the 'server' repo?

[jorgebucaran]

@SkaterDad I think https://github.com/hyperapp/server/issues is the right place, but feel free to comment here or create a new issue in hyperapp/hyperapp/issues if you think it belongs there or it is related to SSR DOM hydrating.

[lyquocnam]

really need a boilerplate hyperapp SSR + ExpressJS with webpack

[frenzzy]

I just open sourced hyperapp-render with SSR and Node.js streaming support :tada:

[marconi1992]

How about this https://github.com/airbnb/hypernova ?

https://github.com/ara-framework/hypernova-hyperapp

[jorgebucaran]

@marconi1992 That's excellent.