Three.js: Texture.clone() reuse of images

[BjornMoren]

I use the Texture.clone() function a lot in my code, because I need to set the UV coordinates differently on every model, and this slows down the initiation of the page considerably. If I don't clone the texture, then the scene renders instantly. So something is not optimal in the rendering pipeline.

It seems that the Texture.clone() function indeed creates a shallow clone that reuses the image, but then when data is sent to the graphics card these two textures are seen as having two separate images, sending the same image twice. Is this correct? Perhaps this is where the bottleneck is.

Coming from XNA and DirectX, where textures are not much more than containers for bitmap data, I wonder why Three.js also includes UV data in them, instead of specifying UV once the texture is applied to a mesh?

[WestLangley]

By "UV data", do you mean texture.offset and texture.repeat?

when data is sent to the graphics card these two textures are seen as having two separate images, sending the same image twice.

Quite likely. What do you get when you type renderer.info.memory into the Console?

[BjornMoren]

WestLangley or any other admin, please contact me on bjorn.moren (at) gmail.com. I have some info about this issue that I want to share in private. Could not find a way to send a private message here on Github. If you don't want to give out your email, then instruct me on how to contact you some other way.

[BjornMoren]

The renderer.info.memory reports the correct number of textures for my scene.

Looking into this I found a more serious problem in the 3D pipeline of either Three.js or WebGL. All I have to do is clone a 1024x1024 texture 1000 times and it will hang my computer. No crash of the browser, just an instant freeze of the system. Dell Inspiron notebook, Win7, Chrome 39. I would say that is a pretty serious exploit given how many users have WebGL enabled.

A temporary work around for the clone problem is to set whatever fields in the original texture that needs to be in the clone, then clone it, then copy the private texture, and NOT set the needsUpdate flag, as follows:

    texture.repeat.set(repeatWidth, repeatHeight);
    var clonedTexture = texture.clone();
    clonedTexture.__webglTexture = texture.__webglTexture;
    clonedTexture.__webglInit = true;
    //clonedTexture.needsUpdate = true;

[BjornMoren]

To play around with the clone problem try the following, see the A, B and C alternatives below.

var viewport = null;
var scene = null;
var camera = null;
var renderer = null;
var controls = null;
var texture = null;
function init()
{       
    // Set up rendering objects
    viewport = document.getElementById("viewport");
    scene = new THREE.Scene();
    camera = new THREE.PerspectiveCamera(75, viewport.offsetWidth / viewport.offsetHeight, 100, 10000);
    renderer = new THREE.WebGLRenderer({antialias:true});
    renderer.setSize(viewport.offsetWidth, viewport.offsetHeight);
    viewport.appendChild( renderer.domElement );
    renderer.setClearColor(0x000000, 1); 
    var ambientLight = new THREE.AmbientLight(0xffffff, 1.0); 
    scene.add(ambientLight); 
    // Camera controls
    controls = new THREE.OrbitControls(camera);
    controls.damping = 0.2;
    controls.addEventListener("change", render);
    camera.position.z = 2000;
    // Load textures
    texture = new THREE.ImageUtils.loadTexture("CrateTexture.jpg", null, addModels); 
}
function addModels()
{   
    var geometry = new THREE.BoxGeometry(100, 100, 100, 1, 1, 1);
    var material = new THREE.MeshBasicMaterial({ map:texture, side:THREE.DoubleSide }); 
    var mesh = new THREE.Mesh(geometry, material);
    mesh.position.x = Math.random() * 1000 - 500; 
    mesh.position.y = Math.random() * 1000 - 500; 
    mesh.position.z = Math.random() * 1000 - 500; 
    scene.add(mesh);
    render();
    // Add geometry
    for (var i = 0; i < 300; i++)
    {
        // A. No cloning
//        var clonedTexture = texture;
        // B. Cloning
//        var clonedTexture = texture.clone();
//        clonedTexture.needsUpdate = true;
        // C. Cloning, but with a hack to not copy the bitmap more than once
        var clonedTexture = texture.clone();
        clonedTexture.__webglTexture = texture.__webglTexture;
        clonedTexture.__webglInit = true;

        var geometry = new THREE.BoxGeometry(100, 100, 100, 1, 1, 1);
        clonedTexture.wrapS = THREE.RepeatWrapping;
        clonedTexture.wrapT = THREE.RepeatWrapping;
        clonedTexture.repeat.set(0.5 + Math.random(), 0.5 + Math.random());
        var material = new THREE.MeshBasicMaterial({ map:clonedTexture, side:THREE.DoubleSide }); 
        var mesh = new THREE.Mesh(geometry, material);
        mesh.position.x = Math.random() * 1000 - 500; 
        mesh.position.y = Math.random() * 1000 - 500; 
        mesh.position.z = Math.random() * 1000 - 500; 
        scene.add(mesh);
    }

    animate();
    render();
    console.log( "Texture count: " + renderer.info.memory.textures   );
}
function animate() 
{
    requestAnimationFrame(animate);
    controls.update();
}
function render()
{
    renderer.render(scene, camera);
}

[WestLangley]

This has come up before on stackoverflow.

[mrdoob]

It seems that the Texture.clone() function indeed creates a shallow clone that reuses the image, but then when data is sent to the graphics card these two textures are seen as having two separate images, sending the same image twice. Is this correct? Perhaps this is where the bottleneck is.

That's indeed the problem. It's on my list of things to fix :)

[zaykho]

Any news ?

I'm using the r75 with my main project, and I absolutely require Sprite + Font texture atlas + TextSprite to render the UI in a very aggressive way and I can't use html/CSS or any other component due to the legacy and modding compatibility with an already existing game.

And with multiple text (300+ characters sprite) + UI design sprite + the user that can type and send text in the 3D scene, the actual Texture.clone() functionality just make the performance collapse and the FPS drop is extreme.

I have already tried many options and other feature, but they have always some drawback that make them unusable in this project.

So, any news or new idea coming up to deal with that problem ?

[mattdw]

@zaykho if you're not needing dynamic lighting or shading, or you're comfortable writing the shader code for that yourself, it's reasonably simple to write a shader that has its own uniforms for offset and repeat, allowing you to share a texture. It's not a universal solution, but it sounds like it might be sufficient in your case.

[zaykho]

@mattdw I don't need dynamic lighting or shading (also, most of those textures will be rendered on top Z position with Orthographic Camera), but, I never wrote shader code, so I will give try and see if the performance is still here with that method.

Is this shader solution will work great in mobile platform though (either performance and compatibility) ?

[tschw]

@BjornMoren

Looking into this I found a more serious problem in the 3D pipeline of either Three.js or WebGL. All I have to do is clone a 1024x1024 texture 1000 times and it will hang my computer. No crash of the browser, just an instant freeze of the system.

Can't blame Three.js for the freeze - it's running in a restricted environment. So worst thing that is possibly _supposed to_ happen is that the browser shuts us down asking for too much memory. As far as WebGL is concerned at all, the specified behavior is context loss.

Excess memory use in the case you describe is a known issue. The current workaround is to remove the Image objects from the textures before saving the scene and adding a reference to the same image manually at load time.

EDIT: Oh wait! I see the renderer uploads the image multiple times, so there is no workaround. I was thinking of the very similar issue that the same texture image ends up multiple times in exports of scenes set up like this one.

[mattdw]

@zaykho this sprite shader is what I'm using; may or may not work for you, but I'm using it for mobile games at 60fps. It's an ES6 module, so you'll have to rewrite it if that's not appropriate.

[zaykho]

@mattdw Thank you !! I will test it now.

Also, for reference purpose, I will post this here too: https://github.com/mrdoob/three.js/issues/5876#issuecomment-81274615

Moving offset to material, we would need 20 cloned materials and one texture.

In fact, moving offset to sprite, we would need 1 material and 1 texture.

^ Still, It would be great to have a proper fix/way into the three.js core about this performance issue.

[MEBoo]

texture.repeat.set(repeatWidth, repeatHeight);
var clonedTexture = texture.clone();
clonedTexture.__webglTexture = texture.__webglTexture;
clonedTexture.__webglInit = true;
//clonedTexture.needsUpdate = true;

Hi @BjornMoren ,
does this trick works anymore?

[MEBoo]

Hi @WestLangley and @mrdoob ...
yes this come up before on stack overflow but now something has changed (r98).

The property __webglTexture doesn't exists anymore, and using the .clone() method, seems cloning the source texture buffer too (I am reading renderer.info.memory).

Don't know if the "logical" texture is cloned and in renderer.info.memory appears as a new texture but the source texture buffer is reused, or if (as it seems) everything is duplicated.

[Mugen87]

The property __webglTexture doesn't exists anymore

You can request the WebGLTexture object like so:

const properties = renderer.properties;
const textureProperties = properties.get( texture );

console.log( textureProperties.__webglTexture );

[MEBoo]

@Mugen87 thanks! But I miss something...

I need to do exately what @BjornMoren did:

var clonedTexture = texture.clone()
clonedTexture.__webglTexture = texture.__webglTexture;
clonedTexture.__webglInit = true;
//clonedTexture.needsUpdate = true;
clonedTexture.offset.set(Math.random(), Math.random());

but if I can get the original __webglTexture like you say, how could I set it to the cloned one?

UPDATE
Don't know if wrong but I did this:

const textureProperties = renderer.properties.get( texture );

var clonedTexture = texture.clone();
renderer.properties.get(clonedTexture); //force the creation of the properties for the new clonedTexture
for (var key in textureProperties )
     renderer.properties.update(clonedTexture,key,textureProperties[key]);

clonedTexture.offset.set(Math.random(), Math.random());
//clonedTexture.needsUpdate = true;  //no need it but doesn't change the final result

Now I have always the same renrerer.info.memory.textures value, and the offset updated for each texture instance.

[Mugen87]

Even if it works, your approach is still a hack and not recommended. You might have evil side effects in your app by doing this.

[adevart]

Is there currently a workaround for this issue? I can't get the above suggestions to work copying the properties over. I am running into the same problem loading spritesheets in ThreeJS:

https://stackoverflow.com/questions/57426845/how-to-load-texturepacker-spritesheets-in-threejs

There was a note on StackOverflow that said the original texture had to be uploaded to the GPU first, which said to use render.setTexture(originalTexture) but I can't see where that function is:

https://stackoverflow.com/questions/25514730/cloning-textures-without-causing-duplicate-card-memory-in-three-js

The images won't show without setting needsUpdate to true and if I do that, it duplicates the source image, which for 2k spritesheets with dozens of animation frames uses multiple GBs of RAM.

Can the texture be forced to display without setting needsUpdate to true, which creates a new WebGL image buffer?

[adevart]

Ok, I got it worked out. I did need to preload the WebGL texture first before assigning it to the other textures. I updated the Stackoverflow page with what I did.

https://stackoverflow.com/questions/57426845/how-to-load-texturepacker-spritesheets-in-threejs/57432562#57432562

Is there a better way to preload a webGL texture than manually creating one and writing an image loaded by THREE.ImageLoader into it?

Does anyone know what the render.setTexture(originalTexture) function mentioned above refers to?

[adevart]

Is Three.Image still in development? I thought the image component of texture was a Three.Image instance (like the type returned by https://threejs.org/docs/#api/en/loaders/ImageLoader ) and could be shared between different texture instances while the texture instances could have unique parameters.

If it's a new class, do you know when this might be ready? I need it for a commercial project in the next 3 months but if it won't be ready, I will use one of the workarounds mentioned earlier or use custom geometry/shader.

[Mugen87]

@adevart No, THREE.Image has not yet landed in dev. It's probably best for your use case to stick to your current workarounds.

[mattdesl]

For those stumbling on this, here's what I did in a recent version of ThreeJS ([email protected]) to ensure single WebGL texture with multiple THREE textures, for a sprite sheet.

// one texture for a large PNG atlas
const atlas = new THREE.TextureLoader().load('atlas.png', () => {
  renderer.initTexture(atlas);
})

// each sprite has its own texture instance
function createSprite () {
  const map = new THREE.Texture();

  // copy over the WebGL handle
  assignTextureHandle(map, atlas);

  // here you can assign per-sprite texture.repeat / offset
  // ... texture.repeat.set(...);

  return new THREE.Sprite(new THREE.SpriteMaterial({
    map
  }))
}

function assignTextureHandle (map, atlas) {
  const atlasData = renderer.properties.get(atlas);
  if (!atlasData.__webglTexture) renderer.initTexture(atlas);
  const mapData = renderer.properties.get(map);
  Object.assign(mapData, atlasData);
}

[Mugen87]

Thanks for sharing! I hope we can focus on #17949 in the next time so such workarounds won't be necessary anymore 😇 .

[adevart]

For those stumbling on this, here's what I did in a recent version of ThreeJS ([email protected]) to ensure single WebGL texture with multiple THREE textures, for a sprite sheet.

Thanks, does this work across different rendering contexts? I tried a method assigning the webglTexture and if I referenced the same webglTexture in a second webGL context like another canvas or renderTexture, it wouldn't render the texture. It only worked in a single context at a given time.

[Mugen87]

Thanks, does this work across different rendering contexts?

Nope, you can't share WebGL resources (e.g. WebGLTexture, WebGLBuffer) across different WebGL contexts.