React-three-fiber: ✍️ Roadmap for 2.x (suggestions welcome)

[drcmda]

• [x] Stable interaction layer so that this lib will be able to interface with react-with-gesture
  
  
  
  • Raytracer order problem: https://github.com/mrdoob/three.js/issues/16031
  
  
• [x] Proper event bubbling, e.stopPropagation()
• [x] Expose renderloop, so that libs like react-spring do not have to run their own (bad for perf)
• [x] Allow frameloop invalidation so that it only renders when there's a need for it
• [x] Stable camera treatment, default-camera rigs, multi-scene rendering, hud, etc.
  
  
  
  • Scene-cam threejs issue: https://github.com/mrdoob/three.js/issues/15929
  
  
  • Working impl draft: https://github.com/drcmda/react-three-fiber#heads-up-display-rendering-multiple-scenes
  
  
• [x] Better non-object3d primitive handling
  
  
  
  • Ref problem: https://github.com/facebook/react/pull/15021
  
  
  • name="xyz" -> attach="xyz"
  
  
  • attachArray="passes" -> allowing objects to fill parent arrays
  
  
• [x] Figure out a definitive set for hooks, currently useThree, useRender
• [ ] Make auto-canvas-measurement optional
• [ ] Tests, with help from @bl4ckm0r3
• [ ] Typescript, with help from @ksjogo

[nickpolet]

These all look great. Very excited to see these progress. This lib is absolutely fantastic to use.

Particularly interested in #2, which is about exposing the renderloop. Would this allow for performance optimisations by having control over when to render?

react-three-renderer has properties on the root object that allow you to control when to render frames or not. It has a forceManualRender property and onManualRenderTriggerCreated. This is incredibly useful as it allows fairly fine grained control and allows you to have a fairly lightweight application that doesn't chug through frames if not needed. But react-three-renderer doesn't work with react@>16.

Is there any way of doing this in the currently library? I've spent a while with react-three-fiber and am loving it. Great work btw.

[drcmda]

Would this allow for performance optimisations by having control over when to render?

Both libs are driven by a requestAnimationFrame-loop. Having two isn't as good as having one. This lib would basically allow any other lib to update by calling foreignLib.update().

react-three-renderer has properties on the root object that allow you to control when to render frames or not. It has a forceManualRender property and onManualRenderTriggerCreated. This is incredibly useful as it allows fairly fine grained control and allows you to have a fairly lightweight application that doesn't chug through frames if not needed.

That will be easy to implement. I added it to the list. Currently you can do useRender, which i think is a fantastic way for components to opt into (or take over) the render phase, but the actual loop is still not exposed. I'll do this with a prop on Canvas that allows you to form your own.

[nickpolet]

That will be easy to implement. I added it to the list.

Amazing! Super useful for reducing wasted CPU cycles. Hello battery life.

[samburgers]

This is an incredible project. Not sure if this is the right thread, but one feature I would certainly make great use of is to be able to dial up common cameras configs (ie isomorphic camera) with ease.

[drcmda]

@samburgers, with the last commits that went in you can do this, easily. Just keep configs in state and write it directly into the cam. You could also switch cams reactively.

function Main() {
  const camera = useRef()
  const { width, height } = useThree().size
  const [config, update] = useState({ fov: 45 })
  return (
    <>
      <perspectiveCamera
        ref={camera}
        aspect={width / height}
        radius={(width + height) / 4}
        {...config}
        onUpdate={self => self.updateProjectionMatrix()}
      />
      {camera.current && <group>{/* ... */}</group>}
    </>
  )
}

[Kelier]

In order to raise the utilization rate, we could have an official website !

[ryanking1809]

Somewhere, where we could start building more thorough documentation would be great as well. I will probably build my own reference as I learn, but I would be great to have a place to share it and have others contribute.

[drcmda]

@Kelier @ryanking1809 would be amazing, but my time's super limited. Would you want to help with that?

[ryanking1809]

@drcmda I'm going to play around with the library over the next week or two, and if it's the right fit I'll be happy to help where I can :) It appears to be exactly what I'm looking for, but I don't want to make any promises then disappear on you.

I think we could get a pretty quick landing page together based off the examples you've already build and improve it from there. I've never set up any documentation tools before, but if we can something off the ground I'd be more than happy to contribute as I learn.

[drcmda]

Sounds good to me. If you find things in your evaluation that you need, you can ping me here - i'm open to adding functionality. I do this to drive a cad-type application @ work later on with drill bits and tool paths, but i want to get the foundation straight first.

[ryanking1809]

Oh cool, I'm trying to build a massive interactive chart / planning tool, and this is the only way I can utilise the GPU - but it also gives me a lot more room to have fun with the visuals so I'm excited!

Looks like I'm going to have a lot of questions around the basics but maybe I'll start collating them together for some sort of docz.site repo.

[ksjogo]

I will be mainly interested in using to drive some electronic card/board games, seems like a a good coverage of different things.

Dragndrop will definitely be something needed for that, what's your imagined API to react-gesture?

Will try to make progress with typing over the week.

[drcmda]

@ksjogo all it needs to function is onEnter onMove onLeave or something like that. I need to figure out a way for it to support more targets than just the dom. I think i'll pick react-springs multi platform api for that, with exposed endpoints like react-with-gesture/three.

Letting libs handle that part would be a major benefit over doing it with events (onDragStart, onDragMove, onDragEnd) because that way gestures can be a re-usable thing, and they're usually more complex than just filling out raw events. The api would remain the same, like this would be a dragndrop:

const [bind, { local: [x, y] }] = useGesture()
return <mesh {...bind()} position={[x, y, 0]} />

I plan to have a story for that this week. We also need it quickly.

[Kelier]

@Kelier @ryanking1809 would be amazing, but my time's super limited. Would you want to help with that?

I can't guarantee enough time I have, but I will try my best to pay attention to and help it.

[rodolfovilaca]

I have a problem that a majority of users in my application uses browser versions that doesn't support WEBGL.

I couldn't find a way to change the Three.js renderer. Is there any possibility of changing the renderer, like from WebGLRenderer to SVGRenderer without causing breaking changes? Could it be a improvement to change the renderer if its not supported yet?

[drcmda]

@rodolfovilaca "canvas" just wraps around the reconcilers render method, which works just like react-doms, or react-natives. It renders into a scene, doesn't actually care which threejs renderer you use to make that visual.

import { render } from 'react-three-fiber'

const renderer = new THREE.SVGRenderer()
renderer.setClearColor(0x000000)
renderer.setSize(window.innerWidth, window.innerHeight)
document.body.appendChild(renderer.domElement)
const camera = new THREE.PerspectiveCamera(75, window.innerWidth / window.innerHeight, 0.1, 1000)
camera.position.z = 35
const scene = new THREE.Scene()

render((
  <mesh>
    <sphereGeometry name="geometry" args={[1, 16, 16]} />
    <meshBasicMaterial name="material" />
  </mesh>
), scene)

You loose some of the functionality, though, like useRender, useThree, events. This stuff is all covered in canvas. Maybe some of it can be shipped into the reconciler, though things like events --- i would smash cross platform goals. Most of these would be very easy to write out, though.

[drcmda]

Here's a running example with both a render loop and useRender implemented: https://codesandbox.io/s/yq90n32zmx useThree wouldn't be much more than a context provider.

[drcmda]

@nickpolet your request is in, it's working. Most of the time it's fully automatic, every time any prop is changed through react it invalidates the render-loop. There will be cases in which you have to give it a ping manually, for instance when you're working with orbitcontrols or your own useRender mutations.

// This will activate loop invalidation, which is automatic most of the time
<Canvas invalidateFrameloop ... />

Sometimes you must be able to kick off frames manually, for instance when you're dealing with async stuff or camera controls:

  // Picking up invalidate from useThree means we only invalidate *this* canvas
  const { invalidate } = useThree()
  const texture = useMemo(() => {
    // Textures load async, it *needs* to invalidate on load, or else there
    // is a race condition once invalidateFrameloop is set to "true"
    const texture = loader.load(url1, invalidate)
    texture.minFilter = THREE.LinearFilter
    return texture
  }, [url1])

You can try in the latest beta

[ryanking1809]

@drcmda I think I'm pretty confident I'll move forward with react-three-fiber. Happy to get started with some documentation. How would you prefer to approach it?
I could just set up a quick docz repo for now, and perhaps you could just fork it, or do whatever you choose in the future?

[drcmda]

@ryanking1809 great to hear, docz would be good choice. Do we want to ship this thing off to the react-spring org? I could make you a contributor with access rights.

[ryanking1809]

Yeah, alright. If you set up an empty repo there, I can get started.
Maybe we can pin something up here to request other contributors as well!

[drcmda]

Here's the repo: https://github.com/react-spring/react-three-fiber-site

[nickpolet]

@nickpolet your request is in, it's working. Most of the time it's fully automatic, every time any prop is changed through react it invalidates the render-loop. There will be cases in which you have to give it a ping manually, for instance when you're working with orbitcontrols or your own useRender mutations.

// This will activate loop invalidation, which is automatic most of the time
<Canvas invalidateFrameloop={true} ... />

Sometimes you must be able to kick off frames manually, for instance when you're dealing with async stuff or camera controls:

  // Picking up invalidate from useThree means we only invalidate *this* canvas
  const { invalidate } = useThree()
  const texture = useMemo(() => {
    // Textures load async, it *needs* to invalidate on load, or else there
    // is a race condition once invalidateFrameloop is set to "true"
    const texture = loader.load(url1, invalidate)
    texture.minFilter = THREE.LinearFilter
    return texture
  }, [url1])

You can try in the latest beta

Just tested this with our codebase and it works perfectly. Thank you for the quick implementation time. We've now managed to optimise rendering of our component with a lot of flexibility.

[crubier]

Have a way to ender / export the scene to Json / GLTF and other formats ? I am not sure if this is feasible already, but that would be really useful on my side, including on server side !

[drcmda]

@crubier i think you're looking at plain threejs for things like that. i'm sure there would be a way. Otherwise, you can of course represent the vdom as a json structure, or server render it, then hydrate. i've never tried but React should be able to do this.

[samburgers]

If touch support is not already on the roadmap I would add a plus one for that!

[drcmda]

@ryanking1809 is working on it, there’s a or already. Touch support will def come.

[drcmda]

@samburgers touch support is in, out in the latest beta. the docs in the 2.0 branch are up to date, they explain the events in detail.

[samburgers]

I have so much love for this project.