Three.js: Addressing ThreeJS's load time: smaller meshes = faster loader (OpenCTM 2?)

[bhouston]

Description of the problem

The main issue with Three.JS these days I find is the slow mesh load time. This is caused primary by the size of the assets - meshes and images. We've recently done an analysis of the ThreeJS binary format and compared it with OpenCTM. We have found on average that OpenCTM meshes are 2x to 5x smaller than ThreeJS's binary format gzipped (e.g. default browser compression) -- which translates into huge improvements to start-up time.

The reason that OpenCTM produces smaller files is because it uses a novel compression method (data re-ordering, and data precision reduction, and other neat things) combined with the superior LZMA compression method (via LZMA.js). I have checked and LZMA is only responsible for about 1/2 to 1/3 of OpenCTM's superior compression ratio, thus the data massaging that OpenCTM does is pretty valuable.

There are issues through with OpenCTM though that prevent it from being a replacement for ThreeJS's binary format: it is no longer maintained (hasn't been updated in 4 years), its index channel doesn't support non-triangle index data (no support for non-indexed vertices, indexed triangle fans, indexed polygons, points, indexed lines), it doesn't support submeshes (i.e. ranges for multi-materials), it doesn't support arbitrary extra channels (only 4 x float channels.)

Arbitrary extra channels needed for minimum parity with ThreeJS's binary format are:

• skin weights (4x float) - supported
• skin bone indices (4 x index)
• morph positions (3 x float)
• morph normals (3 x float)
• colors (3 x float)
• bones (16 x float)
• point sizes (1 x float)
• tangents/binormals (4 x float) - supported

I was wondering if it may be possible to write a new advanced compressed format that exposes the data transforms that improve compression (but maybe doesn't require LZMA? so that one can use brotli, etc.) but doesn't have extremely inflexible hard coded limitations that the current approach has.

My feeling is that one could create a new file format, one that isn't based on any hard coding of the fields. Thus we do not have to anticipate its exact uses, rather we just have to make a flexible API.

Thus I wouldn't hard code any particular fields (indices, vertices, normals, colors, etc.) but instead just allow for names and raw data types (Float32Array, UInt8Array, etc.) along with the stride and hints that enable optimal compression. These hints wouldn't be perscriptive of the compression method (.e.g inflexible) but rather would be descriptive of the data (let the library make the best choice of compression method, if any) -- and thus it is up to the library to figure out which compression and data transforms to use to achieve the high compression of OpenCTM. When reading, one would just read by name and you'd get the uncompressed data back - the data format will completely specify how to read the data and untransform it.

We could have a set of prebuild hints for the standard fields (normal, position, skin weights, indices, etc.), thus using it in a basic way would not require any real understanding of its details.

This would ensure we can save anything we want in this format, even if we do not anticipate it now. We could likely store animations in this file format in the future. I would write the compressor and decompressor in JavaScript but I would keep the compressor as simple and small as possible so that it can be easily ported to other languages like OpenCTM was.

Further details: At the bottom of this page is the specifications of the OpenCTM file format and its API: http://openctm.sourceforge.net/?page=about

If @tschw wanted a challenge that makes use of your skills that greatly moves this project forward, at least from my perspective (you'd have to check with @mrdoob), this would be it. I think that the design given above would have long-term viability and it really does address all of OpenCTM's many current shortcomings. I would have wide usefulness outside of ThreeJS as well.

Three.js version

• [x] Dev
• [x] r76

• [ ] ...

  Browser

• [x] All of them

• [ ] Chrome
• [ ] Firefox

• [ ] Internet Explorer

  OS

• [x] All of them

• [ ] Windows
• [ ] Linux
• [ ] Android
• [ ] IOS
  

  Hardware Requirements (graphics card, VR Device, ...)

[bhouston]

BTW @tschw I am not saying you should or must do this, just that I think it is something that could be insanely valuable and it seemed to be like the stuff you like. I was also thinking of writing this myself or having another member of our team contribute this. What is nice about this is that it would make huge differences in start-up time -- 2x or more. I think there is very few other improvements in ThreeJS world (and beyond) that could have such long-lasting and wide improvements than this.

[tschw]

@bhouston

I am not saying you should or must do this, just that I think it is something that could be insanely valuable and it seemed to be like the stuff you like

Can you provide some kind of funding or implementation bounty? I still have time until the end of the month and it would really help me right now :). Starting next month, I'll be doing non-WebGL stuff, so my activity around here may become limited to maintenance mode for a while.

There are issues through with OpenCTM though that prevent it from being a replacement for ThreeJS's binary format: it is no longer maintained (hasn't been updated in 4 years), its index channel doesn't support non-triangle index data (no support for non-indexed vertices, indexed triangle fans, indexed polygons, points, indexed lines), it doesn't support submeshes (i.e. ranges for multi-materials), it doesn't support arbitrary extra channels (only 4 x float channels.)

For compression, there's also Open3DGC, supported by glTF.

[ghost]

OBJ, JSON, DAE and all the other TEXT files are not suitable for online projects.
One huge text file and dozens of image files used as textures is the last thing you want to download.
The best option for THREE.js is SEA3D.
https://github.com/sunag/sea3d
Binary format and LZMA compression.

[bhouston]

@tschw, I can offer you some funding immediately to do an analysis with me of options, even if it doesn't lead to an implementation (which could happen as well.) Hit me up on Skype: ben.exocortex

[makc]

unrelated xkcd strip

[sunag]

I implemented Open3DGC for SEA3D with encoder in C++ and App. The robot.sea with 1.107MB now + Open3DGC only 652kB. Really good compression algorithm. I hope to do this with the animations too.

http://community.poonya.com/224

[arose]

Hi, this is not directly about geometry compression but might be of interest to you.

I am currently involved in developing a binary and compressed format for on-line use of macro-molecular structures (proteins, DNA). We found MessagePack as a container format plus bespoke compression techniques to work quite well. The MMTF format can easily handle structures with 2 million atoms (that is coordinates plus metadata), parsing them in < 0.5 seconds. You can read more about it at http://mmtf.rcsb.org and in the format specification.

[bhouston]

Very interesting development from Google: https://github.com/google/draco

[bhouston]

I'd advocate Three.JS adopt Draco or Open3DGC as its mesh compression format.

[ikerr]

FYI, we did a performance comparison of Google's Draco vs. Khronos's Open3DGC:

https://docs.box.com/v2.0/blog/showdown-3d-model-compression-at-scale-draco-vs-open3dgc

[valette]

FYI, we did a performance comparison of Google's Draco vs. Khronos's Open3DGC:
https://docs.box.com/v2.0/blog/showdown-3d-model-compression-at-scale-draco-vs-open3dgc>

interesting, but it seems to me that there is a quantization issue here : encoding coordinates with 11 bits indeed increases compression ratios, but should significantly degrade visual quality, especially on the most complex models ( the dragon and the statuette ). Have you inspected the visual quality of the decompressed models?

[ikerr]

Hi @valette, it could be that some detail is lost, but the dragon does seem to pass the "eyeball" test. It would be interesting to measure the error introduced by the compression using some sort of error metric, but I haven't gotten around to that yet. It would also be interesting to see if both algorithms respond similarly to different quantization levels, or if the comparison changes.