Three.js: NodeJS support

[crubier]

Is your feature request related to a problem? Please describe.

Running Three on Node is a massive pain

Making packages that are based on Three and meant to run on Web, NodeJS, and React Native is a massive pain.

With a colleague we have plan to make it easier by providing a node specific build, just like there is a jsm specific build.

Describe the solution you'd like

The plan is to do this:

• Create node compatible versions of at least:
  
  
  
  • src/loaders/ImageLoader.js => src/loaders/ImageLoader.node.js
  
  
  • src/loaders/FileLoader.js => src/loaders/FileLoader.node.js
  
  
  • examples/js/loaders/GltfLoader.js => examples/js/loaders/GltfLoader.node.js
  
  
  • examples/js/loaders/GltfExporter.js => examples/js/loaders/GltfExporter.node.js
  
  
• Edit the rollup config of utils/build/rollup.config.js :
  
  
  
  • Add an alias plugin to manage the two first changed packages above
  
  
  • Add a fourth bundle of three with output: "three.node.js" including the alias plugin
  
  
• Edit the utils/build/modularize.js:
  
  
  
  • Add another dst folder additionally to the jsm one
  
  
  • Add a field called nodePath in files items pointing to the node version of the file if relevant
  
  
  • Modify the convert function to output something similar to the jsm version with: if ( keys ) imports.push( 'import {${keys}\n} from "${pathPrefix}../../build/three.node.js";' );
  
  

Would you accept such a PR back into the main three.js repo / package ?

Describe alternatives you've considered

We use a custom sh*y fork of an old version three in our repo and it is messy, we'd like to go for a cleaner version that is now possible thanks to the cleanup of the three codebase using ES modules.

Additional context

[mrdoob]

Running Three on Node is a massive pain.

Can you elaborate?

• Create node compatible versions of at least:
  
  
  
  • src/loaders/ImageLoader.js => src/loaders/ImageLoader.node.js
  
  

Can you show how ImageLoader.node.js would be different from ImageLoader.js?

[crubier]

Can you elaborate?

Actually it is more using three to build "agnostic" packages that are meant to run on node, react native, and web which is a massive pain.

We built a lot of relatively complex 3D geometry/Mesh processing and similar packages on top of three r109, and we import these packages in apps that run on node, web or react native, and having a version of three that does not crash on any of the three platform was a very delicate process, needing very specific tweaks in the code AND in the bundlers.

Now we want to update three and all that goes out the window, so we would like to have a cleaner and more reproducible approach.

This might be an instance of the x y problem, so let me know if there turns out to be an easy way to do that, which I'd be very happy to hear about. From my perception, it seemed like a common wisdom that three was hard on non-web platforms, since even the docs says using three on node can be complicated

Can you should how ImageLoader.node.js be different from ImageLoader.js?

In our current crappy version, it involves conditionally using jsdom to create a makeshift DOM to pass to the image loader on node. It also involves importing fs to load the image data from local relative paths, and injecting the data inside the image element, and other dirty tricks to deal with metro bundler, webpack, rollup and other bundlers. It has reached a point of non-return were the code is bonkers, but works (surprisingly) for all our use cases.

In this project, I'd like to take a much more sane approach, in a file that is guaranteed to only be run on node js, and no other platforms.

[crubier]

For reference, here is our current crazy implementation of ImageLoader for three 109 (Pretty bad I know):

import { Cache, DefaultLoadingManager } from "three";

import { document, window, navigator } from "../dom";
import { startsWith, isNil } from "lodash/fp";
import pathModule from "path";
let fs = null;
function ImageLoader(manager) {
  this.manager = manager !== undefined ? manager : DefaultLoadingManager;
}

Object.assign(ImageLoader.prototype, {
  crossOrigin: "anonymous",

  load: async function(url, onLoad, onProgress, onError) {
    if (url === undefined) url = "";

    if (this.path !== undefined) url = this.path + url;

    //////////////////////////////////////////////////////////////////////////////////////////////////
    // START: The modification compared to normal image loader
    if (startsWith("data:", url)) {
      console.log("IS datauri");
    } else {
      console.log("IS not datauri ");
      if (isNil(fs)) {
        let fsModule = await import("fs-extra"); // Done once only
        // Trick to solve problems caused by various packagers adding layers of "default" nesting on exports...
        // It is needed for our combination of rollup for packages and metro for the react native app... 
        // eslint-disable-next-line no-prototype-builtins
        while (fsModule.hasOwnProperty("default")) {
          fsModule = fsModule.default;
        }
        fs = fsModule;
      }
      // const realPath = pathModule.resolve(url);
      const imageData = await fs.readFile(url);
      url = `data:image/jpeg;base64,${imageData.toString("base64")}`;
    }
    // END: The modification compared to normal image loader
    //////////////////////////////////////////////////////////////////////////////////////////////////

    url = this.manager.resolveURL(url);

    const scope = this;

    const cached = Cache.get(url);

    if (cached !== undefined) {
      scope.manager.itemStart(url);

      setTimeout(function() {
        if (onLoad) onLoad(cached);

        scope.manager.itemEnd(url);
      }, 0);

      return cached;
    }

    const image = document.createElementNS(
      "http://www.w3.org/1999/xhtml",
      "img"
    );

    function onImageLoad() {
      image.removeEventListener("load", onImageLoad, false);
      image.removeEventListener("error", onImageError, false);

      Cache.add(url, this);

      if (onLoad) onLoad(this);

      scope.manager.itemEnd(url);
    }

    function onImageError(event) {
      image.removeEventListener("load", onImageLoad, false);
      image.removeEventListener("error", onImageError, false);

      if (onError) onError(event);

      scope.manager.itemError(url);
      scope.manager.itemEnd(url);
    }

    image.addEventListener("load", onImageLoad, false);
    image.addEventListener("error", onImageError, false);

    if (url.substr(0, 5) !== "data:") {
      if (this.crossOrigin !== undefined) image.crossOrigin = this.crossOrigin;
    }

    scope.manager.itemStart(url);

    image.src = url;

    return image;
  },

  setCrossOrigin: function(value) {
    this.crossOrigin = value;
    return this;
  },

  setPath: function(value) {
    this.path = value;
    return this;
  }
});

export { ImageLoader };

And here is the dom file which is imported above (Probably one of the craziest file in our codebase, which is mostly pretty sane aha):

import { isNil } from "lodash/fp";

let realWindow = null;
let realDocument = null;
let realNavigator = null;
let isBrowserReal = false;
let isNodeReal = false;
const jsdomOptions = {
  pretendToBeVisual: true,
  storageQuota: 1e9,
  resources: "usable",
  runScripts: "dangerously"
};
// Trick packagers to NOT find the jsdom import in order to not resolve it at build time and let the try catch handle errors at runtime
// 🤡 Yes it sounds like a joke but this is the only solution we found that works on all 3 platforms with all bundlers.... 
const toto = {
  coco: require
};
const generateJsdomModuleName = () => "js" + "dom";
const jsDomName = generateJsdomModuleName();
// const jsDomName = "js" + "dom";
let jsdomValue = null;

try {
  try {
    //@ts-ignore
    if (isNil(window)) {
      if (isNil(jsdomValue)) {
        // 🤡 the second part of that sad clownesque joke...
        jsdomValue = toto.coco(jsDomName);
      }
      const { JSDOM } = jsdomValue;
      realWindow = new JSDOM(
        `<!DOCTYPE html><body id="main"><p >Hello world</p></body>`,
        jsdomOptions
      ).window;
      realDocument = realWindow.document;
      realNavigator = realWindow.navigator;
      isNodeReal = true;
    } else {
      isBrowserReal = true;
      //@ts-ignore
      realWindow = window;
      realDocument = realWindow.document;
      realNavigator = realWindow.navigator;
    }
  } catch (e) {
    if (isNil(jsdomValue)) {
      jsdomValue = toto.coco(jsDomName);
    }
    const { JSDOM } = jsdomValue;
    realWindow = new JSDOM(
      `<!DOCTYPE html><body id="main"><p >Hello world</p></body>`,
      jsdomOptions
    ).window;
    realDocument = realWindow.document;
    realNavigator = realWindow.navigator;
    isNodeReal = true;
  }
} catch (e2) {
  realWindow = {};
  realDocument = {};
  realNavigator = {};
}

export const window = realWindow;
export const document = realDocument;
export const navigator = realNavigator;
export const isBrowser = isBrowserReal;
export const isNode = isNodeReal;

[crubier]

As you see our main problem originates from trying to use ONE version of three for 3 platforms.

This is why we'd like to take a different approach, starting with having one sane version of three per platform.

[donmccurdy]

I like the goal of taking a systematic approach to node.js support. I've been hoping (but am not sure) that the changes can be mostly focused into key files like FileLoader and ImageLoader, and that most files, like GLTFLoader, would not need separate compilation outputs. I realize this would require a few changes to GLTFLoader itself. It would also be really valuable to have unit tests running in node.js, in addition to in a browser.

Some of the work might be simplified by importing browser globals from a common file. For example:

// src/.../platform.js
export let _document = document;
export let _window = window;

if (IS_NODEJS) {
  _document = { ... };
  _window = { ... };
}

// examples/jsm/loaders/GLTFLoader.js
import { _document, _window, ... } from 'three';

[crubier]

@donmccurdy Yes this is the direction we took in our current version. However I am wondering if we really need to keep this emulation of DOM.

Ideally I'd like to avoid having to deal with the bulkiness of JSDom, even if this means losing access to some part of Three first. But it's probably way harder without it, nd not worth it. Happy to hear opinions here.

[donmccurdy]

I didn't mean to suggest emulation of the DOM -- just consolidating references to globals that might differ across environments, so that Node.js support doesn't need to touch so many files.

[crubier]

Yes I agree that this aspect is useful.

In practice, as you wrote, it is mostly about some form of emulation of the DOM though, but I agree it's probably the best option to have good coverage of Three in Node.

[donmccurdy]

Sorry I'm unclear — I do not think we should emulate DOM APIs at all. By DOM APIs I mean the browser's interface to HTML. This is mostly what JSDom does, and shouldn't be necessary for the parts of the library most users should need in Node.js.

Providing other APIs that the browser would normally provide — like TextEncoder/TextDecoder, XmlHttpRequest, atob/btoa, and so on, is probably more critical (and hopefully easier). That could be done either by patching the global namespace in Node.js, or by ensuring the library imports from a common place, which can be "swapped" when running in Node.js, similar to the import example above.

[crubier]

Yes indeed this is a good idea. The only place where it is still problematic is the fact that Images (for textures and others) are moved around in Three JS as instances of the DOM's img element. So that img element (at least, as far as I know) needs to be emulated. Obviously canvas too but this is a different problem, only for rendering.

But overall yes absolutely, architecturally, I would much prefer importing the useful elements from a single place rather than using globals. Bbut that would impact a lot of files in the codebase. Would @mrdoob agree with such a large change (importing window, document, navigator explicitly) in many files just to accommodate one use case ?

[crubier]

Updated plan:

• For Three core:
  
  
  
  • Create a file called dom-globals.js
    
    
    
    
    
    • Create a file called dom-globals.js that exports dom globals used in three js codebase
    
    
    
    • Create a file called dom-globals.node.js that polyfills and exports dom globals for node js
    
    
    
    • Go through all files in three js core codebase and explicitly import globals from dom-globals.js
    
    
    
  
  
  • Edit the src/Three.js file to export all from dom-globals.js (to make them available to examples files, see later)
  
  
  • Edit the rollup config of utils/build/rollup.config.js :
    
    
    
    
    
    • Add a fourth bundle of three with output: "three.node.js" including the alias plugin
    
    
    
    • Add an alias plugin to alias dom-globals.js to dom-globals.node.js when building for node
    
    
    
  
  
  • Optional, if needed: Create node specific compatible versions of:
    
    
    
    
    
    • src/loaders/ImageLoader.js => src/loaders/ImageLoader.node.js
    
    
    
    • src/loaders/FileLoader.js => src/loaders/FileLoader.node.js
    
    
    
  
  
• For the example folder:
  
  
  
  • Go through all files in three js example codebase and explicitly import globals from three
  
  
  • Edit the utils/build/modularize.js:
    
    
    
    
    
    • Add another dst folder additionally to the jsm one
    
    
    
    • Add a field called nodePath in files items pointing to the node version of the file if relevant
    
    
    
    • Modify the convert function to output something similar to the jsm version with: if ( keys ) imports.push( 'import {${keys}\n} from "${pathPrefix}../../build/three.node.js";' );
    
    
    
  
  
  • Optional, if needed: Create node specific compatible versions of the following (and add their file path to utils/build/modularize.js)
    
    
    
    
    
    • examples/js/loaders/GltfLoader.js => examples/js/loaders/GltfLoader.node.js
    
    
    
    • examples/js/loaders/GltfExporter.js => examples/js/loaders/GltfExporter.node.js
    
    
    
  
  
• For the test suite:
  
  
  
  • Copy test/unit/three.source.unit.js to test/unit/three.source.node.unit.js and comment all tests first, adding them progressively and fixing problems when they appear
  
  
  • Edit the /Users/vincent/Code/three.js/test/rollup.unit.config.js and add a configuration similar to the one with input: 'unit/three.source.unit.js', but with input: 'unit/three.source.node.unit.js', with the alias plugin with the same settings as above. In this file, add some node-specific tests (FileLoader tests for node for example)
  
  
  • Edit three.js/test/package.json to add a new test script using qunit in node ( https://qunitjs.com/intro/ ), testing the new 'unit/build/three.source.node.unit.js'
  
  
  • Edit three.js/package.json to add a "test-e2e-node": "node test/e2e/node.js" which includes a e2e test scenario like: load a local GLTF, transform it a bit, apply a texture to it, and write it locally.
  
  

[etiennedupont]

Hey folks,

I just submitted a PR implementing the points @crubier mentioned above. It's all work in progress since I did not add tests but at least there can be now a separate bundle of three for node along with associated examples. I'd like to have your opinion on the DOM elements that I could have missed. So far I checked for the following globals:

export const Blob = window.Blob;
export const atob = window.atob;
export const btoa = window.btoa;
export const DOMParser = window.DOMParser;
export const document = window.document;
export const XMLHttpRequest = window.XMLHttpRequest;
export const TextEncoder = window.TextEncoder;
export const TextDecoder = window.TextDecoder;
export const decodeURIComponent = window.decodeURIComponent;
export const CustomEvent = window.CustomEvent;

The current tests suite is passing with the newly created node version (relying on JSDOM) which is not very surprising, but since I now explicitly export the DOM elements from src/dom-globals.js, the tests are failing on the regular version, when run through "npm test". Now I plan to add some non trivial tests on highly DOM dependant objects of the node version (like FileLoader) and see how it goes.

[mrdoob]

What's wrong with emulating/polyfilling browser APIs in node?

[crubier]

Nothing wrong really.

Using JSDOM to polyfill browser APIs has the advantage of being simple and easy to maintain. This is the way we are going right now, and probably the easiest one. Basically slapping JSDOM on top of Three and be happy.

My initial idea was to have some deeper integration with NodeJS in some places of the Three codebase (for example loading files using the Node FS api directly instead of using JSDOM + "file:///" URIs). But I think we are not going to go this way in the end.

One problem though with the JSDOM solution is that it needs JSDOM to be registered as a (Peer? Optional?) dependency of Three in the package.json. What's your opinion on this?

[mrdoob]

Using JSDOM to polyfill browser APIs has the advantage of being simple and easy to maintain. This is the way we are going right now, and probably the easiest one. Basically slapping JSDOM on top of Three and be happy.

I'm confused. Didn't you say it was a massive pain?

[mrdoob]

Running Three on Node is a massive pain

Ah, I can see you retracted that now.

[mrdoob]

I think it would be better to invest these efforts in improving the documentation for people that want to run the library on node.

[crubier]

I think it would be better to invest these efforts in improving the documentation for people that want to run the library on node.

The problem is there is no way to do it cleanly today, the only way is to stuff jsdom globals into node globals, which is clearly advised against by JSDOM: https://github.com/jsdom/jsdom/wiki/Don't-stuff-jsdom-globals-onto-the-Node-global . It feels like and is a "dirty hack".

The changes in this MR ( #20924 ) allow to have a clean and official way to run ThreeJS in Node. For example as of today, there are no tests of FileLoader in the Three Codebase because it cannot be tested automatically. This MR allows to run Three cleanly in Node, test it properly and so on. This MR only add a small maintenance cost IMO.

[mrdoob]

From: https://github.com/jsdom/jsdom/wiki/Don't-stuff-jsdom-globals-onto-the-Node-global

A common antipattern we see when people use jsdom is copying globals from a jsdom window onto the Node.js global, and then trying to run the code---intended for a browser---inside Node.js. This is very bad and you should not do it.

I'm confused. Wasn't the whole point of node.js to allow developers use the same code and libraries in front-end and back-end? Why aren't they providing the same APIs browser provide? How's deno dealing with this?

[crubier]

Wasn't the whole point of node.js to allow developers use the same code and libraries in front-end and back-end?

Yes mostly: Allow to create libraries in the same language for both Frontend and Backend. But it does not mean that there is 100% overlap in their capabilities:

• There are common points since both use v8: https://ecma-international.org/ecma-262/9.0/#sec-ecmascript-standard-built-in-objects
• Browsers have APIs that only them have: https://developer.mozilla.org/en-US/docs/Web/API

• Node has its own APIs that browsers don't have: https://nodejs.org/docs/latest/api/

  I made a quick diagram to explain this:

The problem today is that ThreeJS uses some (not a lot) functions that are only available in the browsers.

The good thing is that JSDOM is a library for node which emulates most browser APIs.

Why aren't they providing the same APIs browser provide?

Mostly because they are different systems with different needs:

• Browser do not allow direct filesystem access, for obvious security reasons. On the other hand, server side code needs access to the file system.
• Node JS is often used to create HTTP servers so it contains specific functions for this, which is useless in a browser.
• The notion of a "Document" is obvious in a web page on a browser, but is meaningless in a program runnning on a server.

How's deno dealing with this?

Same as Node. v8 Engine + additional APIs, but critically, does not include most Web APIs.

[mrdoob]

Thanks for the clarification 🙏

So then my question is... Wasn't JSDOM created to solve this?

[crubier]

So then my question is... Wasn't JSDOM created to solve this?

Yes it was, but the current state of the ThreeJS codebase does not allow to use it correctly, as stated above, because the ThreeJS codebase expects to have Web APIs in the global window object.

From there, we have the choice between three options:

• Keep the current status, which is far from ideal IMHO as I stated.
• Correct this situation with the smallest possible impact: This is the PR #20924
  
  
  
  • Add a self-contained build of ThreeJS that can be used on NodeJS directly (which is just the standard ThreeJS + JSDOM to polyfill what is needed)
  
  
  • Add a build of examples for NodeJS
  
  
  • Add tests that should have been done earlier but could not be added because of the limitations of Three in NodeJS (See here).
  
  
• A more ambitious approach that I wanted to do initially, which modified the NodeJS compatible version of ThreeJS a bit more, to allow to directly interface with node filesystem API for example and others. This would allow a more seamless use of ThreeJS in Node JS, but would add some maintenance burden so we decided against it, because we respect maintainers and we hope this PR to get merged 😅

[crubier]

@mrdoob we have a new minimalistic version of the PR coming with:

• Only 4 additions:
  
  
  
  • additions of 2 optional peer dependencies in package json, for people who want to run three in node, they need jsdom + canvas
  
  
  • a new file src/window.js with polyfills for node js using jsdom + canvas
  
  
  • addition of one build configuration in utils/build/rollup.config
  
  
  • additions in utils/modularize.js to generate examples versions compatible with node js, just like there is the jsm version today
  
  
• New tests to tests things that were not tested before, which adds 12 additions

So one question: should we:

• Create 2 separate PRs? (1 PR with just the 4 necessary changes + 1 PR with the 12 additions of tests) ?
• Or just 1 PR with all the changes?

[mrdoob]

Could this be done in a separate repo/package? three-node or something?

[etiennedupont]

@mrdoob of course it is feasible, as a fork of your repo. Since our goal is to have several equivalent builds of three for different platform, it seems more natural and less prone to bug (by versions mismatch for instance) to keep all builds at the same place. Moreover, as @crubier was saying, that node version would allow to complete most "to do" tests without having them failing on node. @donmccurdy was also suggesting that it would be valuable to have unit tests running on node.js.

Example of one of the unit tests I added in test/unit/src/loaders/FileLoader.tests.js, that passes on both node.js and browser versions:

        QUnit.test( "load regular url => json", ( assert ) => {

            const done = assert.async();
            const fileLoader = new FileLoader();
            fileLoader.responseType = "json";
            fileLoader.load(
                "https://raw.githubusercontent.com/mrdoob/three.js/dev/examples/models/json/suzanne_buffergeometry.json",
                // onLoad callback
                function ( data ) {

                    // output the text to the console
                    // console.log( data );
                    assert.ok( data.metadata.position === 505, "Passed!" );
                    done();

                },

                // onProgress callback
                function ( xhr ) {

                    console.log( ( xhr.loaded / xhr.total * 100 ) + '% loaded' );

                },

                // onError callback
                function ( err ) {

                    console.error( 'An error happened' );
                    console.error( err );
                    assert.ok( false, "Failed with error" );
                    done();

                }
            );

        } );

Example of one of the end to end node.js tests I was able to add:

import { GLTFExporter } from "../../../examples/node/exporters/GLTFExporter";
import { BoxBufferGeometry, MeshBasicMaterial, Mesh, Scene, TextureLoader } from "../../../build/three.module.node";
import { writeJson } from "fs-extra";

export default function main() {

    const exporter = new GLTFExporter();
    const textureLoader = new TextureLoader();
    textureLoader.load( `file://${__dirname}/../../data/image.jpg`, function ( texture ) {

        const scene = new Scene();
        const box = new Mesh(
            new BoxBufferGeometry(),
            new MeshBasicMaterial( { map: texture } )
        );
        box.name = "box-test";
        scene.add( box );

        exporter.parse( scene, function ( gltf ) {

            const path = `${__dirname}/outputs/scene-exporter.gltf`;
            writeJson( path, gltf, {}, function ( data ) {

                console.log( "Scene written" );

            } );

        } );

    }, undefined, function ( error ) {

        throw error;

    } );


}

[mrdoob]

@mrdoob of course it is feasible, as a fork of your repo.

No no, I didn't mean a fork.

I meant a different repo which adds this one as a dependency and has the additional scripts and generated files.

For every release, the maintainer of three-node will have to run the script to generate the node.js-friendly files.

[crubier]

Just to be clear, the approach you propose has the following tradeoffs:

• Pros:
  
  
  
  • Maintenance of the scripts/config to build ThreeJS for NodeJS is not done by ThreeJS maintainers (changes to 4 files)
  
  
  • You will avoid getting some Issues opened due to the fact that JSDom is not perfect and a few ThreeJS examples do not work on NodeJS (AudioContext and ImageBitmap not implemented in JSDom)
  
  
• Cons:
  
  
  
  • Several functions remain untested in ThreeJS codebase today, while they would be tested cleanly in our PR, they will not be testable with the current approach.
  
  
  • No unification, there will always be some kind of a crutch to run ThreeJS in NodeJS instead of a simple official solution
  
  
  • Makes it more difficult for users like us who want to create cross-platform packages, we have to conditionally import two different packages depending on our target. For example we'd like to use react-three-fiber in NodeJS, but we will have to find a hack so that react-three-fiber imports three-node on node...
  
  
  • The NodeJS version will be a bit more shitty to maintain than it could have been, because the separate repo using ThreeJS as a dependency will consist mainly of some weird magic find-and-replace scripts and re-exports, instead of a clean rollup config in our PR.
  
  

We'd like your confirmation before we go in that direction

[mrdoob]

Yep, lets go in that direction.

Once the repo is setup and we have a clear picture of what's needed and the side-effects, we can reconsider the setup.

Thanks!

[crubier]

It seems that we can't publish under the names three-node nor three-node-js, we'll try to find another name...

[etiennedupont]

@mrdoob here's the repo implemented:
https://github.com/sterblue/three-universal