Babylon.js: Any interest in shared KTX 2.0 parser?

[donmccurdy]

Discussed this briefly with @bghgary recently, but I think the current state of (non-WASM) KTX 2.0 parsers for web is:

• three.js has a custom plain JS parser, THREE.KTX2Loader. Parses KTX2 wrapper and decodes using MSC Transcoder.
• babylon.js has a TypeScript port KTX2 Decoder. Parses KTX2 wrapper using combination of MSC Transcoder and lightweight AssemblyScript transcoders.
• (new) I've started a TypeScript KTX2 parser + serializer, ktx-parse. It's a standalone repository with an npm package, so that I can use it in other projects like glTF-Transform. My intention (open to discuss) was to keep it focused on parsing the container, not transcoding, so that it could be used with any of the various Basis transcoders or other GPU texture formats.

three.js does not bring in production dependencies from NPM, so my ability to package that code for reuse is a bit limited. If you're interested though, I'd be glad to collaborate on a repo/package for the other two, and for future users. Currently the code I've written in read.ts is very similar to your own ktx2FileReader.ts, and outputs a nearly-identical container interface. I've added support for key/value data, otherwise I think they're functionally equivalent. I intend to include serialization as well (not functional yet), which will be tree-shakeable if you don't need it:

import { read, write } from 'ktx-parse';

// Load.
const data = await fetch('./input.ktx2').then(r => r.arrayBuffer());

// Parse.
const container = read(new Uint8Array(data));

// Serialize.
write(container); // → Uint8Array

Open to ideas on how to structure this, or let me know if this doesn't make sense. :)

[sebavan]

As we are also not having dependencies, I ll definitely let @bghgary and @Popov72 think about this one, but I truly enjoyed the shared aspect and community focus of the proposal !!!

[bghgary]

Yeah, we don't bring in npm dependencies either. Would it be possible to create a standalone js file that we distribute? For example, Draco has the js file directly commited in their repo and we just copy it. Another example is glTF-Validator where we use browersify to build a standalone js.

That is what we were thinking for the KTX2 Decoder. It would live in its own repo and we would bring in a pre-built js file. Open to other ideas though.

[Popov72]

Currently the code I've written in read.ts is very similar to your own ktx2FileReader.ts

That's expected as we heavily based this implementation on your own implementation from 3js KTX2Loader.js (this is our first reference in the header comment of ktx2Decoder.ts) ;)!

[donmccurdy]

Would it be possible to create a standalone js file that we distribute?

That's easy if it's just a parser+serializer, yes. If scope includes transcoders, ZSTD or ZLIB decompression, and Web Worker management, then those could add several extra JS+WASM files that need to be copied around in the right directory paths, and things get more complicated.

[bghgary]

I don't think we should include web worker management. I think this should be handled by the consuming code.

I also don't think we should include the transcoders or ZSTD directly. Instead, we can use configuration options to point to them or add them via some kind of extensibility mechanism. It will be up to the consuming code to determine where to host the additional assets (wasm, etc.).

[donmccurdy]

I've finished up the reading/writing functionality of ktx-parse now, and this quick transcoding test works:

const fs = require('fs');
const savePixels = require('save-pixels');
const ndarray = require('ndarray');
const {read} = require('ktx-parse');

const decoderWASM = fs.readFileSync('./node_modules/universal-texture-transcoders/build/uastc_rgba32_srgb.wasm');
const ktxData = fs.readFileSync('./test_uastc.ktx2');

(async () => {
    const texture = read(ktxData);

    const {byteLength} = texture.levels[0].levelData;

    // Uncompressed texture padded to multiple-of-4 height
    const yBlocks = (texture.pixelHeight + 3) >> 2;
    const uncompressedByteLength = texture.pixelWidth * yBlocks * 4 * 4;

    const texMemoryPages = (65535 + byteLength + uncompressedByteLength) >> 16;
    const memory = new WebAssembly.Memory({ initial: texMemoryPages + 1 });

    const decoder = (
        await WebAssembly.instantiate(decoderWASM, {env: {memory}})
    )['instance'].exports;

    const compressedTextureView = new Uint8Array(memory.buffer, 65536, byteLength);
    compressedTextureView.set(texture.levels[0].levelData);

    const decodedTextureView = new Uint8Array(memory.buffer, 65536 + byteLength, uncompressedByteLength);
    const status = decoder.decodeRGBA32(texture.pixelWidth, texture.pixelHeight);

    if (status !== 0) throw new Error('Decoding failed.');

    saveTofile('output.png', decodedTextureView, texture);
})();

///////////////////////////////////////////////////////////////////////////////

async function saveTofile(path, view, texture) {
    const pixels = ndarray(view, [texture.pixelWidth, texture.pixelHeight, 4])
        .transpose(1, 0)
    const image = await new Promise((resolve, reject) => {
        const chunks = [];
        savePixels(pixels, 'png')
            .on('data', (d) => chunks.push(d))
            .on('end', () => resolve(Buffer.concat(chunks)))
            .on('error', (e) => reject(e));
    });
    fs.writeFileSync(path, image);
}

I didn't include ZSTD decompression in the test, but it should be minor to add:

import { ZSTDecoder } from 'zstddec';
const decoder = new ZSTDDecoder();
await decoder.init();

// ...

if (texture.supercompressionType === 2) {
    data = decoder.decode( data, texture.levels[i].uncompressedByteLength );
}

The build output is https://unpkg.com/[email protected]/dist/ktx-parse.modern.js, and would benefit from tree-shaking if you don't need to write KTX2 files. Does this fit what you had in mind? I