Three.js: Revert FileLoader-based ImageLoader

[mrdoob]

The initial idea was to gain support for progress events in ImageLoader but the side effects have been so far:

1. Security issues. #9826
2. data: urls no longer working. https://github.com/mrdoob/three.js/commit/0576c7c80c472beb67c855031975aa7708175370
3. Some people (A-Frame) relied on image.src being a file path for their tests.
4. CrossOrigin not working. #10168
5. SVG no longer loading. #10363

So, for the time being, I think it'll be better to revert and, instead, try to convince W3C people to provide load progress events for Image.

[mrdoob]

So, for the time being, I think it'll be better to revert and, instead, try to convince W3C people to provide load progress events for Image.

Seems like Adobe already tried 4 years ago...

http://blogs.adobe.com/webplatform/2012/01/13/html5-image-progress-events/
http://blogs.adobe.com/webplatform/2012/07/10/image-progress-event-progress/

[mrdoob]

/cc @dmarcos

[makc]

ha ha. fwiw, I once had to replace image based loader with xhr based loader exactly to make cross origin textures work in old safari - xhr did work before images got crossOrigin attribute there. so, quoting 'CrossOrigin not working' as a reason for revert is funny )

[RemusMar]

So, for the time being, I think it'll be better to revert and, instead,
try to convince W3C people to provide load progress events for Image.

+1

Seems like Adobe already tried 4 years ago...

The W3C people are interested in much more "important" things.

[stevenvergenz]

Is this slated for r84 then? We run a special VR browser, and we can't use blob URLs, so the sooner this is resolved the happier we'll be.

[mrdoob]

@stevenvergenz yep!

[dhritzkiv]

Should the documentation be updated for ImageLoader and TextureLoader to reflect the lack of onProgress callback support? And maybe a code comment in the commented out progress event handler block?

[mrdoob]

/ping @looeee

[dhritzkiv]

Also, what exactly are the conditions that: a request _can_ be made for image data; have the image's src set with a blob from createObjectURL; and maintain progress events? Would it be worth having as a case within ImageLoader?

(I realize that the previous version had something along these lines, so apologies if this has been discussed ad nauseam)

Something like this untested code:

//cover as many conditions that don't work via the blob route
if (
    //not an svg
    !/\.svg$/.test(this.path) &&
    //not a local file
    !(
        //the path starts with file
        /^file:\/\//.test(this.path) ||
        //or it's a relative path, and the current document's protocol is "file"
        (!/:\/\//.test(this.path) && /^file:\/\//.test(location.protocol))
    ) &&
    //not a data uri
    !/^data/.test(this.path)
) {
    //proceed to load it in as a blob
    var loader = new FileLoader();

    if ( onProgress ) onProgress( event );

    loader.setPath( this.path );        
    loader.setResponseType( 'blob' );       
    loader.setWithCredentials( this.withCredentials );

    loader.load( url, function ( blob ) {

        image.src = URL.createObjectURL( blob );    

    }, onProgress, onError ); 
} else {

    //otherwise load it in the current way (aka what the code was reverted to)

}

I don't understand all the gotchas involved, but I know it was working for my case (PNG/JPEGs served from a third party CDN over HTTPS – I don't know what the crossOrigin situation was with that), so it's a shame to lose progress callback functionality.

I have a feeling we'll be waiting a long time for standardized Image progress events.

[looeee]

/ping @looeee

I'll have time in a couple of days to update the docs 😄

[jpiar]

where did you revert to? both r83 and r84 are missing setWithCredentials method in both ImageLoader and TextureLoader when compared to r82. This caused a major problem in loading images.

[mrdoob]

r84 is the release that goes back to the old ImageLoader. Until we figure out a better solution, you can overwrite the loader in your project like this:

THREE.ImageLoader = function ( manager ) {

    this.manager = ( manager !== undefined ) ? manager : THREE.DefaultLoadingManager;

}

Object.assign( THREE.ImageLoader.prototype, {

    load: function ( url, onLoad, onProgress, onError ) {

        var scope = this;

        var image = document.createElementNS( 'http://www.w3.org/1999/xhtml', 'img' );
        image.onload = function () {

            image.onload = null;

            URL.revokeObjectURL( image.src );

            if ( onLoad ) onLoad( image );

            scope.manager.itemEnd( url );

        };
        image.onerror = onError;

        var loader = new THREE.FileLoader();
        loader.setPath( this.path );
        loader.setResponseType( 'blob' );
        loader.setWithCredentials( this.withCredentials );
        loader.load( url, function ( blob ) {

            image.src = URL.createObjectURL( blob );

        }, onProgress, onError );

        scope.manager.itemStart( url );

        return image;

    },

    setCrossOrigin: function ( value ) {

        this.crossOrigin = value;
        return this;

    },

    setWithCredentials: function ( value ) {

        this.withCredentials = value;
        return this;

    },

    setPath: function ( value ) {

        this.path = value;
        return this;

    }

} );

[jpiar]

Thank you MrDoob.

[jbreckmckye]

I came across this issue when trying to add progress events for my textures.

My solution was to load the file via FileLoader first, then load from the cache with a TextureLoader. In-memory caching means the file is only loaded once, even if it doesn't get cache headers from the server.

The result is that we get progress events via FileLoader's initial AJAX fetch, but can still exploit all the useful behaviour of the proper TextureLoader (such as disabling alpha channels for JPEG textures, and other optimisations).

/**
 * Loads THREE Textures with progress events
 * @augments THREE.TextureLoader
 */
function AjaxTextureLoader() {
    /**
     * Three's texture loader doesn't support onProgress events, because it uses image tags under the hood.
     *
     * A relatively simple workaround is to AJAX the file into the cache with a FileLoader, create an image from the Blob,
     * then extract that into a texture with a separate TextureLoader call.
     *
     * The cache is in memory, so this will work even if the server doesn't return a cache-control header.
     */

    const cache = THREE.Cache;

    // Turn on shared caching for FileLoader, ImageLoader and TextureLoader
    cache.enabled = true;

    const textureLoader = new THREE.TextureLoader();
    const fileLoader = new THREE.FileLoader();
    fileLoader.setResponseType('blob');

    function load(url, onLoad, onProgress, onError) {
        fileLoader.load(url, cacheImage, onProgress, onError);

        /**
         * The cache is currently storing a Blob, but we need to cast it to an Image
         * or else it won't work as a texture. TextureLoader won't do this automatically.
         */
        function cacheImage(blob) {
            // ObjectURLs should be released as soon as is safe, to free memory
            const objUrl = URL.createObjectURL(blob);
            const image = document.createElementNS('http://www.w3.org/1999/xhtml', 'img');

            image.onload = ()=> {
                cache.add(url, image);
                URL.revokeObjectURL(objUrl);
                document.body.removeChild(image);
                loadImageAsTexture();
            };

            image.src = objUrl;
            image.style.visibility = 'hidden';
            document.body.appendChild(image);
        }

        function loadImageAsTexture() {
            textureLoader.load(url, onLoad, ()=> {}, onError);
        }
    }

    return Object.assign({}, textureLoader, {load});
}
module.exports = AjaxTextureLoader;

Thought it might be helpful to share.

[makc]

[Molosc]

Hi @jbreckmckye

Your solution work fine, but just at the first time. When I try to call the same image later with AjaxTextureLoader, I got this error:
Uncaught TypeError: Failed to execute 'createObjectURL' on 'URL': No function was found that matched the signature provided. at cacheImage

Any idea to fix it?

[jbreckmckye]

@Molosc - I think what is happening is that the loader is plucking the data from cache, then passing the cached texture to createObjectURL. But createObjectURL only works with Blob and File objects, not raw binary data (like in a texture).

The solution will be to add a cache check. Could you try out the following and let me know if it works?

/**
 * Loads THREE Textures with progress events
 * @augments THREE.TextureLoader
 */
function AjaxTextureLoader() {
    /**
     * Three's texture loader doesn't support onProgress events, because it uses image tags under the hood.
     *
     * A relatively simple workaround is to AJAX the file into the cache with a FileLoader, create an image from the Blob,
     * then extract that into a texture with a separate TextureLoader call.
     *
     * The cache is in memory, so this will work even if the server doesn't return a cache-control header.
     */

    const cache = THREE.Cache;

    // Turn on shared caching for FileLoader, ImageLoader and TextureLoader
    cache.enabled = true;

    const textureLoader = new THREE.TextureLoader();
    const fileLoader = new THREE.FileLoader();
    fileLoader.setResponseType('blob');

    function load(url, onLoad, onProgress, onError) {
        const cached = cache.get(url);
        if (cached) {
            return cached;
        } else {
            fileLoader.load(url, cacheImage, onProgress, onError);
        }

        /**
         * The cache is currently storing a Blob, but we need to cast it to an Image
         * or else it won't work as a texture. TextureLoader won't do this automatically.
         */
        function cacheImage(blob) {
            // ObjectURLs should be released as soon as is safe, to free memory
            const objUrl = URL.createObjectURL(blob);
            const image = document.createElementNS('http://www.w3.org/1999/xhtml', 'img');

            image.onload = ()=> {
                cache.add(url, image);
                URL.revokeObjectURL(objUrl);
                document.body.removeChild(image);
                loadImageAsTexture();
            };

            image.src = objUrl;
            image.style.visibility = 'hidden';
            document.body.appendChild(image);
        }

        function loadImageAsTexture() {
            textureLoader.load(url, onLoad, ()=> {}, onError);
        }
    }

    return Object.assign({}, textureLoader, {load});
}
module.exports = AjaxTextureLoader;

[duhaime]

@looeee Is the threejs.org site in a public repo somewhere? I'm happy to submit a PR to get these docs updated...

[looeee]

@duhaime there's an 'edit' button in the top right of each docs page, that should get you started.

[Smiley43210]

For anyone who wants the asynchronous file loader from @jbreckmckye in class and Promise/async function flavor, here it is. It is based on his second revision, which uses caching to avoid duplicate HTTP requests. Using cached entries has been fixed. Added for my own use case, to avoid creating separate loaders, it also supports .svg files as well, done by overriding the MIME type of the blob used internally. If there are other file types that require this, they can be added to the mimeOverrides array.

AsyncTextureLoader

// Adapted from https://github.com/mrdoob/three.js/issues/10439#issuecomment-293260145
import * as THREE from 'three';

/**
 * THREE's texture loader doesn't support onProgress events, because it uses image tags under the hood.
 *
 * A relatively simple workaround is to AJAX the file into the cache with a FileLoader, create an image
 * from the Blob, then extract that into a texture with a separate TextureLoader call.
 *
 * The cache is in memory, so this will work even if the server doesn't return a cache-control header.
 *
 * Loads THREE Textures with progress events.
 * @extends THREE.TextureLoader
 */
export default class AsyncTextureLoader extends THREE.TextureLoader {
    constructor() {
        super();
        // Turn on shared caching for FileLoader, ImageLoader and TextureLoader
        THREE.Cache.enabled = true;

        this.fileLoader = new THREE.FileLoader();
        this.fileLoader.setResponseType('blob');
    }

    load(url, onProgress) {
        const mimeOverrides = [
            ['svg', 'image/svg+xml']
        ];

        return new Promise((resolve, reject) => {
            /**
             * The cache is currently storing a Blob, but we need to cast it to an Image or else it
             * won't work as a texture. TextureLoader won't do this automatically.
             * 
             * Use an arrow function to prevent creating a new this binding, which would prevent the
             * usage of super methods.
             */
            const cacheImage = (blob) => {
                // Change the MIME type if necessary
                const extension = url.match(/\.([\w\d]+)$/);
                for (let override of mimeOverrides) {
                    if (extension[1] == override[0]) {
                        blob = new Blob([blob], {type: override[1]});

                        break;
                    }
                }

                // Object URLs should be released as soon as is safe, to free memory
                const objUrl = URL.createObjectURL(blob);
                const image = document.createElementNS('http://www.w3.org/1999/xhtml', 'img');

                const errorHandler = (error) => {
                    reject(error);
                };
                const loadHandler = () => {
                    // Cache the texture
                    THREE.Cache.add(url, image);
                    // Release the object URL
                    URL.revokeObjectURL(objUrl);
                    // Remove image listeners
                    image.removeEventListener('error', errorHandler, false);
                    image.removeEventListener('load', loadHandler, false);
                    document.body.removeChild(image);
                    // Load image as texture
                    super.load(url, resolve, () => {}, reject);
                };

                // Add image listeners
                image.addEventListener('error', errorHandler, false);
                image.addEventListener('load', loadHandler, false);

                image.src = objUrl;
                image.style.visibility = 'hidden';
                document.body.appendChild(image);
            }

            const cacheEntry = THREE.Cache.get(url);
            if (cacheEntry) {
                // Load cached image as texture
                super.load(url, resolve, () => {}, reject);
            } else {
                this.fileLoader.load(url, cacheImage, onProgress, reject);
            }
        });
    }
}

[mrdoob]

Btw, all loaders hace a loadAsync method now:
https://threejs.org/docs/#api/en/loaders/Loader.loadAsync