React-three-fiber: sharing textures/materials/geometries

[greggman]

Sorry if this is noob question but how do I share textures, materials, and geometry?

In normal three.js I can do this

const texture = loader.load('someurl');
const material = new THREE.MeshBasicMaterial({map: texture});
const geometry = new THREE.BoxBufferGeometry(1, 1, 1);

const m1 = new THREE.Mesh(geometry, material);
const m2 = new THREE.Mesh(geometry, material);
const m3 = new THREE.Mesh(geometry, material);
const m4 = new THREE.Mesh(geometry, material);

m1.position.x = -2;
m2.position.x = -1;
m3.position.x = 1;
m4.position.x = 2;

scene.add(m1);
scene.add(m2);
scene.add(m3);
scene.add(m4);

What's the equivalent here?

I get the

<group>
    <mesh position=...>
    <mesh position=...>
    <mesh position=...>
    <mesh position=...>
</group>

part but I'm not sure I get the rest.

in normal DOM React heavy resources are referred to by URLs and the browser magically manages them so

<div>
  <img url="sameUrl" />
  <img url="sameUrl" />
  <img url="sameUrl" />
  <img url="sameUrl" />
</div>

Only actually loads one image into the browser. The browser also knows when that image is safe to free.

It feels like texture, geometry, and materials are kind of the equivalent in three.js but here they're being treated as DOM elements when in fact they are not part of the hierarchy. Like the actual bits of an image they are outside the hierarchy so it seems confusing to me to make them part of the hierarchy.

[drcmda]

You can either stow them away in a useMemo (which right now is the cleanest solution)

const material = useMemo(() => new THREE.MeshBasicMaterial({ color }), [color])
return (
  <mesh material={material} />
  <mesh material={material} />
  <mesh material={material} />
)

Or you do pack it into the tree, and use refs

const ref = useRef()
const [material, set] = useState()
useEffect(() => void set(ref.current), [color])

return (
  <meshBasicMaterial ref={ref} color={color} />
  {material && (
    <mesh material={material} />
    <mesh material={material} />
    <mesh material={material} />
  )}
)

It feels like texture, geometry, and materials are kind of the equivalent in three.js but here they're being treated as DOM elements when in fact they are not part of the hierarchy. Like the actual bits of an image they are outside the hierarchy so it seems confusing to me to make them part of the hierarchy.

They're not dom elements, they're native elements. The reconciler knows they're not part of the actual scene, but it can handle them still, which allows you to let React manage these objects automatically, which normally would pile up like crazy. React will do the cleanup (even calling dispose when they go away).

But they can also write into the scene objects:

<mesh>
  <meshBasicMaterial name="material" /> // Not a Threejs scene object
  <sphereGeometry name="geometry" args={[1, 16 , 16]} /> // Not a Threejs scene object
</mesh>

You can read more about this here: https://github.com/drcmda/react-three-fiber#objects-and-properties

[drcmda]

Sharing is indeed something that isn't as straight forward as i'd like it to be. If you have ideas how to handle this better my ears are open. With a reconciler under our control we can do lots things.

[drcmda]

One idea i had would maybe allow them to call into a render child, if they detect one, so you could do

<meshBasicMaterial color={color}>
  {material => (
    <mesh material={material} />
    <mesh material={material} />
    <mesh material={material} />
  )}
</meshBasicMaterial>

That would solve it, and even take care of deeply nested re-use.

[greggman]

Sorry I don't have a solution at the moment but it does feel a little like "if all I have is a hammer everything is a nail" type of solution right now.

It's certainly convenient to be able to do

<mesh>
  <meshBasicMaterial name="material" /> 
  <sphereGeometry name="geometry" args={[1, 16 , 16]} /> 
</mesh>

but it feels like it makes no sense. The material and geometry are not children of the mesh. They are resources referenced by the mesh. I can't think of any equivalent to the above example in React. There are no resources in React.

[drcmda]

they're not children strictly speaking, but attached to the object in one way or another. if you omit "name" they'll be kept as properties somewhere in the object. you don't have to do it though, you can manage these objects yourself - i just think that this is the main reason three becomes so tiresome so fast, because the overhead of objects to manage goes out of hand quickly. in practice i've found it valuable to let react manage this stuff for me, and perhaps even more efficient - since i don't recreate objects as much as i did before react. also pretty good feeling to know react will dispose them on unmount. But anyway, this is all optional.

[drcmda]

Ps. look at this example for instance: https://github.com/drcmda/react-three-fiber#dealing-with-effects-hijacking-main-render-loop and the live demo: https://codesandbox.io/embed/y3j31r13zz

These are not scene objects. But now that they are part of the v-dom, they become reactive. This also makes it compatible with everything react has to offer. In the demo for instance the effect is regulated via react-spring.

<animated.glitchPass name="passes" renderToScreen factor={factor} />

And if the component would unmount, the effect would simply disappear and all would be like it was before. In three js that alone would have caused so much unwanted complexity and churn, doing this esssentially for free in react is just too good to pass on.

[greggman]

Those are great examples but we're back to the same issue. These objects are not part of the V-DOM and should arguably not be part of the V-DOM. You can go read what DOM is. It has functions like appendChild and removeChild and collection children. Geometry, Texture, Material don't fit

I'm not saying it's not useful but there's clearly an issue. They are being squeezed somewhere they don't belong.

I don't yet believe this is an "either it's done the way you have it now" or "it sucks" position. Maybe there is some other way to do it that actually fits the react model and is just as or even more flexible. Not sure what that is, only pointing out that what's there now clearly doesn't match the way react treats other resources.

[drcmda]

the vdom has nothing to do with the dom. it's badly named, but it's more like a generic virtual tree structure. react simply builds a graph consisting of components and "native elements", which it doesn't know - it merely knows their name. The reconciler later takes care of managing it. there's nothing odd or weird here, maybe if you look at it from threejs'es perspective, but the reconciler has no problem with these objects, they're part of the vdom. You don't need to call "appendChild" or "removeChild", this is merely an optional step. In any case, react will manage the node itself. If you don't append, the object won't ever reach the hosts native representation of reacts internal vdom graph. Though in our case, these objects do get appended, if you use the "name" property. They don't get pushed into "children", but into named properties (the glitchpass above gets added to "parent.passes" for instance) - which is all fine and legal.

The dom btw has a similar concept. There are many tags that don't need to display, for instance audio without controls. They're just part of a logical tree, but not part of the visual tree, the renderer skips them.

[greggman]

I get all of that. But it's a basic tenet of good software design that you don't mix things that are fundamentally different in the same tree of nodes. Yes, an audio tag might be a strange exception except really not. By default an audio node displays audio controls. An audio node not in the DOM tree is well known wart in the browser and in fact wouldn't work in react, react will stick in in the tree even though you don't want it in the tree unless you go to some crazy contortions to prevent it.

A geometry is not a child of a mesh. It's not, period. It should not be represented as one

[drcmda]

It is though. Or at least it's debatable.

const mesh = new Mesh(geo, mat)

adds both geo and mat as props under "geometry" and "material", just like

scene.add(mesh)

adds the mesh to "children". Both are just props and references. The only difference is the name of the prop.

With the reconciler i want to make three reactive, the only possible way to do this is to allow secondary objects to be managed, because they cause much of the complexity. That is the same conclusion the other three reconcilers have come to, like react-three-renderer and so on.

[drcmda]

I made a hook called useResource that abstracts it a little: https://github.com/drcmda/react-three-fiber/blob/2.0/readme.md#useresourceoptionalrefundefined

[raiskila]

I'm wondering if there's an accepted pattern for sharing "disposable" objects like geometries across multiple instances of the same component. Let's say my <Teapot> component consists of a mesh that renders the same geometry every time. Am I correct in assuming that using a single BufferedGeometry would improve performance over creating a new geometry for each teapot?

The useResource() hook allows us to share objects intra-component, but not across multiple instances of the same component. We would need to make creating the teapot geometry the responsibility of the parent component and have it pass the geometry down in props to share it across <Teapot>s.

To ensure any objects that get created are disposed of when the tree unmounts (particularly in concurrent mode), we can't create disposable objects during the render phase (i.e. in function component bodies or inside useMemo). The solutions that come to mind are some kind of <ObjectCache> component higher up in the tree that exposes a context-based API mimicking useMemo, but it makes sure to only ever create or dispose of objects inside useEffect() and returns null on first render. Or one could use some kind of reference-counted object cache outside React that should only be accessed in useEffect().

[drcmda]

yes, that's right. merged buffergeom is a lot faster. you can put these objects into global state, app state, etc.

for disposal you can do useEffect. but i agree, having some declarative, re-usable wrap around it is probably for the best.

[brainkim]

I’m thinking about declarative scene graphs and reusing geometries and materials with elements and came across this issue. Have y’all considered adapting some of the patterns used by SVG to define and share nodes? For instance there’s <defs>, which allows you to define svg nodes without rendering them, giving them unique id attrs. And then you can reference them with a href attr via special elements like <use>, <clipPath>, and <textPath>.

Something like this id/href system would be very useful for not having to directly manage Three.js objects while still sharing them, and would solve the mismatch of scene graphs and element trees.

[drcmda]

it could be done in userland even today i think. the problem is name bleeding and well, there's no official concept for this in react. you could always make a group, put stuff in there and share the ref via context. but on the first dry-run it's gonna be empty because react renders in one pass whereas svg probably renders defs first, then the rest. this can also be fixed but not sure if it makes sense to make this a construct of the r3f lib.

[brainkim]

ID collisions is definitely an issue here. SVG allows you limit the scope of ids with objects/iframes/shadow dom stuff but it’s all very dubious. As I investigate this stuff I just think that having an exact mapping of Three.JS classes to VDOM nodes is not the best approach, even if it’s the most transparent. Like... I always seem to want to use buffer geoms over the regular geoms, and I’m still not sure about having meshes attach geometries and materials via the parent child relationship of JSX when they seem to act much more like attrs/props than regular children. I’m looking into other xml extensions for inspiration but things like a-frame seem way too high-level.

Anyways, just some thoughts I had, nothing is necessarily actionable. Thanks for the response!

[drcmda]

i agree, i think for now the best is to make it declarative where it makes sense and otherwise leave the imperative as it is:

const [useStore] = create(set => ({
  geometries: {
    mesh1: new THREE.BufferGeometry(...),
    ...
   }
})

// ..

const g = useStore(state => state.geometries.mesh1)
<mesh geometry={g} />

suspense and caching is also very interesting, we'll see how that turns out. useLoader imo works very well as it does currently and being able to manage async assets has many upsides. it also completely takes care of stuff having to to use the loaded assets. could later turn into something that makes sharing assets easier.