Three.js: 2D Text Sprites

[mscongdon]

I'd like to render simple text strings that "always face the camera". To save overhead, 2D text would be ideal. Unfortunately, I have not found any features in Three.js that support this. I don't want to put the text into an image [sprite] because I need to update/change it dynamically.

Please prove my searches insufficient and point me to an example that illustrates this. It would greatly enhance our user experience!

Cheers,
Michael

[WestLangley]

http://mrdoob.github.com/three.js/examples/webgl_geometry_text.html

[mscongdon]

I've been using that for 3D geometry and it looks nice. But it's overkill when I just need some simple, 2D text. Is there any 2D text (or text sprite equivalent) in Three.js?

I plan to have 20-30 elements of text that change each time I update my animation (continuously). Think of a stop watch that continuously shows you the number of milliseconds,seconds, minutes, etc. throughout the animation). Creating 20+ 3D text elements would be exponentially more CPU-intensive than a 2D approach. It already turned my animation from smooth to jittery when I tried to do this with just 1 element of text in 3D.

[chandlerprall]

You can overlay normal HTML elements on top of the 3D canvas.

[mscongdon]

If I were to take a guess at that, it would be more like having game scores at the top of the canvas regardless of what changes within the canvas...right? I'd need this text to move within the canvas. Imagine multiple people walking around a scene and, instead of a light bulb over their head, they have some text that changes frequently (like the timer example I mentioned above). Text in the distance would be smaller than text in the foreground. Someone walking left-to-right would need their text to move left-to-right with them. Can Three.js support that?

[mrdoob]

"A picture is worth a thousand words" :)

[mscongdon]

Do you think I can reduce the 3D overhead dramatically and implement this feature if I still used TextGeometry, but simply do not extrude the text more than 1 pixel...and continuously translate and rotate each text element to face the camera? Or does the TextGeometry use the same amount of CPU whether it creates a 1-pixel deep-object vs. a 20-pixel-deep object?

[chandlerprall]

That's still a lot of processing. You can calculate where an object is in the 2D space of the canvas and position the text, in an HTML element, at that point, changing the size based on the object's distance to the camera. To my knowledge this is how games such as Runescape display player names in the world.

[mrdoob]

1312 has a better snippet for projecting 3D vectors.

[BKcore]

Or you can draw your text on a 2D canvas with fillText(), and use that canvas as a texture for your billboard sprites.

Or with the same technique, render all your texts on a canvas 2D, then blend that texture in a postprocess pass.

[talktough]

@mscongdon - this is exactly what I am trying to do at the moment as well.

I need to put label annotations on a canvas next to objects that are being rotated by the user. I also need to be able to detect events when either the objects or text are clicked by the user.

I managed to get a piece of text next to an object using canvas text and then using 3D to 2D conversion of the coordinates as outlined above. However I can't work out how to rotate the text or detect click events on them.

When I was using Flash I used to remove all the texts and then add them back at the new positions for every animation frame which worked ok but I can't work out an equivalent here for canvas text.

Just wondering if you have you made any progress ?

Thanks

Chris

[mscongdon]

I'll be working this over the next week and will update this thread with my findings. Thanks, everyone, for your suggestions so far!

[mscongdon]

The code samples (including issue #1312) are helpful to get the 2D coordinates. However, I still am unclear how to put 2D text onto a canvas that's already using the WebGLRenderer.

I tried putting 2 renderers the same canvas like this:

var canvas = document.getElementById('myCanvas');
var renderer = new THREE.WebGLRenderer({canvas: canvas});

// do all of my 3D stuff

var renderer2D = new THREE.CanvasRenderer({canvas: canvas});
renderer2D.domElement.getContext('2d').fillText("Hello", 50, 50);

The renderer2D.domElement.getContext('2d') returns null

If I reverse my code to initialize my renderer2D before my renderer, then I can get the '2d' context just fine, but the WebGLRenderer initialization fails with the error Error creating WebGL context.

Hence, I cannot appear to get both a webgl (or experimental-webgl) and a '2d' context on the same HTMLCanvasElement to apply both my 3D and my 2D renderings.

I'm not convinced that the API spec at this URL http://www.w3.org/TR/html5/the-canvas-element.html#the-canvas-element says this is NOT possible.

"Returns null if the given context ID is not supported or if the canvas has already been initialised with some other (incompatible) context type (e.g. trying to get a "2d" context after getting a "webgl" context)."

Does the use of experimental-webgl (rather than simply webgl) as the primary context cause '2d' to not work?

Am I close to getting my 2D text sprites working? Or should I be thinking of this entirely differently? Please advise.

[mscongdon]

OK. I figured out a workable solution.

Step 1. Create 2 canvases - One for 3D and One for 2D.

<canvas id="canvas2D" width="500" height="500"></canvas>
<canvas id="canvas3D" width="500" height="500"></canvas>

Step 2. Give both canvases the same dimensions and location, and set the z-index of them so that the 2D canvas is on top of the 3D canvas. And be sure to set the background-color to 'transparent' for the 2D canvas.

#canvas3D{
  position: absolute;
  left: 0;
  top: 0;
  z-index: 1;
}
#canvas2D{
  background-color: transparent;
  position: absolute;
  left: 0;
  top: 0;
  z-index: 2;
}

This essentially treats "canvas2D" like a pane of glass onto which you paint your 2D images, yet still able to see through to the 3D canvas behind it.

Step 3: Lookup both canvases and create both a CanvasRenderer and a WebGLRenderer:

canvas2D = document.getElementById('canvas2D');
renderer2D = new THREE.CanvasRenderer({canvas: canvas2D});
canvas3D = document.getElementById('canvas3D');
renderer3D = new THREE.WebGLRenderer({canvas: canvas3D});

Step 4: Aside from the regular 3D renderings, here is how we start to draw the 2D text sprites. First, calculate the X,Y location where you want to fill the text from some position within your 3D world. I used a variation of the function in Issue #78:

var coord =  toScreenXY(myPosition, camera, canvas3D);

function toScreenXY(position, camera, canvas) {
  var pos = position.clone();
  var projScreenMat = new THREE.Matrix4();
  projScreenMat.multiply(camera.projectionMatrix, camera.matrixWorldInverse);
  projScreenMat.multiplyVector3( pos );

  return { x: ( pos.x + 1 ) * canvas.width / 2 + canvas.offsetLeft,
      y: ( - pos.y + 1) * canvas.height / 2 + canvas.offsetTop };
}

Step 5: Clear the 2D canvas and re-write the text:

ctx2d = renderer2D.domElement.getContext('2d');
ctx2d.clearRect (0, 0, window.innerWidth, window.innerHeight);
ctx2d.fillText("Hello", coord.x, coord.y);

I'm happy to at least get this working. I'll worry about efficiencies or other drawbacks later! :-)

Cheers,
Michael

[mrdoob]

What about using <div>?

var text = document.createElement( 'div' );
text.style.position = 'absolute';
text.innerHTML = 'Oh hai!';
//
text.style.left = coord.x + 'px';
text.style.top = coord.y + 'px';

[firendust]

i had the same text relate 3d objects problem here, and i think mrdoob's "div" solution is much more convenient... :D

[Yona-Appletree]

I'm having a similar problem, but I'm not sure the general "paint your text outside of three.js" will work for me. I want the text to exist inside the 3d world such that occlusion by other objects and scaling with distance work correctly.

Although I haven't yet tried it, it seems that you could create a plane geometry with a texture that was created from another canvas that had text drawn onto it, then every such plane at the camera during each render phase. This seems like a significant amount of work, however, and while I'm rather new to 3d programming, I understand that many engines support having a textured-plane that always faces the camera.

Thoughts?

[mrdoob]

As stated in the guidelines, help requests should be done in stackoverflow. This board is for bugs or feature requests.

[Yona-Appletree]

My apologies. I searched for the problem on google and ended up here, where it appeared there was already a discussion on the topic, so it seemed appropriate. I'll post something on stackoverflow.

[jareiko]

FYI, you can create a 2D text mesh using ShapeGeometry instead of TextGeometry.

This reduces the number of faces generated because it doesn't extrude.

var shapes, geom, mat, mesh;

shapes = THREE.FontUtils.generateShapes( "Hello world", {
  font: "helvetiker",
  weight: "bold",
  size: 10
} );
geom = new THREE.ShapeGeometry( shapes );
mat = new THREE.MeshBasicMaterial();
mesh = new THREE.Mesh( geom, mat );

[natewilliford]

My solution was to use a Particle with the ParticleCanvasMaterial. This exposes the canvas context through the program parameter which you can use to draw your text directly on the canvas.

var material = new THREE.ParticleCanvasMaterial({
  color: 0x000000,
  program: function(context) {
    context.font = "5pt Helvetica";
    context.fillText("Hello World", 0, 0);
  }
});
var particle = new THREE.Particle(material);

I had to tweak the position and scale of the particle to get it to display properly, but it seems to work well. Plus you can treat your text as an object in your scene and position it (etc) like you would any object.

_EDIT:_ This is using the CanvasRenderer. I'm not sure how/if it will work with webGL.

[oleduc]

Canvas as texture is fast enough

var canvas1 = document.createElement('canvas');
    var context1 = canvas1.getContext('2d');
    context1.font = "Bold 40px Arial";
    context1.fillStyle = "rgba(255,0,0,0.95)";
    context1.fillText(text, 0, 50);

    var texture1 = new THREE.Texture(canvas1);
    texture1.needsUpdate = true;

    var material1 = new THREE.MeshBasicMaterial( { map: texture1, side:THREE.DoubleSide } );
    material1.transparent = true;

    var mesh1 = new THREE.Mesh(
        new THREE.PlaneGeometry(canvas1.width, canvas1.height),
        material1
    );

    mesh1.position.set(position.x + 10,position.y,position.z);

    scene.add( mesh1 );

[makc]

What about using <div>?

That actually implies lot of stuff. I.e. parent div must be position:relative and overflow:hidden

[endel]

I know this is an old thread, but I'd like to share a module I've created to draw text from canvas into THREE.Mesh: http://gamestdio.github.io/three-text2d/