Filament: How to render a renderable in different positions of one scene at a time

[flyingonly]

In our practical use, we need to render a renderable several times on one scene with different transforms at the same time. For now, we have tried to load a glb file for multiple times so that we have several filamentAssets at the same time. But it turns out to be a burden on the memory if the glb file has many textures, since each filament asset will load its own buffer and textures.
We wish to have an easy way to render a renderable more than once on one scene at a time, or to copy filamentAssets without load glb for multiple times. Is there a possible solution to this?

[prideout]

We might be able to add a simple instancing feature to gltfio. Out of curiosity how many instances will you be creating, ballpark? Hundreds? Thousands?

[romainguy]

Renderables are fairly lightweight wrappers around vertex buffers, index buffers and material instances. It should be fairly trivial to add an API to at least clone a renderable. That would solve this request without using a ton of memory.

[flyingonly]

We might be able to add a simple instancing feature to gltfio. Out of curiosity how many instances will you be creating, ballpark? Hundreds? Thousands?

Thanks for the reply. For now we just need about a dozen or so instances.

[gpyalt]

@romainguy, Would it be possible instead to allow a renderable to store an optional array of mat4 transforms, which would allow its efficient rendering at various locations using OpenGL instancing?

[romainguy]

Instancing is something we'd like to do at some point but it's more complicated than just having multiple transforms since it could be useful to vary other attributes as well.

[gpyalt]

Yes, that is true. Supporting a transform would be a good start though :-)

[prideout]

I've been thinking about this. Currently, FilamentAsset has unique ownership over vertex buffers, index buffers, and textures. If I changed these to use shared ownership, then I could add a clone() method to FilamentAsset that would create new a set of entities with new renderable & transformable components.

I think this would be a simple solution. Each clone would have unique transforms and material instances, but they would share their materials and vertex buffers.

[MingweiSamuel]

How can I help add this feature?

I see currently it is possible to do this with Filamesh (since it exposes Vertex and IndexBuffer), just not with gtfio, is that right? What changes need to be made in gtfio?

[prideout]

I still think that adding a public clone() method to FilamentAsset is the way to go. On the implementation side, perhaps all Filament objects owned by FilamentAsset could be wrapped in a shared_ptr with a custom deleter.

[MingweiSamuel]

Wouldn't cloning FilamentAsset potentially end up cloning a lot more than necessary? Cloning Renderable instances seems more precise

[prideout]

It would not be a deep clone, the textures etc would be shared, which is why I mentioned the potential use of shared_ptr to help with the implementation. The cloned FilamentAsset would have a separate set of entities, components, and material instances. The textures, vertex buffers, and index buffers would be shared.

[MingweiSamuel]

It just seems like a waste if you need to clone a particular mesh renderable a dozen times that you would also have to clone the texture material instances and light etc handles to just to be thrown out.

For my current use I actually only need to clone Renderables' meshes which would only require making the VertexBuffers and IndexBuffers accessible. But then the question is how to match up the entities (since not every one is necessarily a Renderable) and also things like what PrimitiveType is it ..
https://github.com/google/filament/blob/1292610289ca81f3f667ca3b1ed02b8eb81f8db6/libs/gltfio/src/FFilamentAsset.h#L228-L229

[romainguy]

But you wouldn't clone the textures or lights, just the renderables.

[prideout]

Yes, the Renderable and Transformable components would be cloned (i.e. the entity hierarchy) as well as Material Instances. Textures and buffers would gain shared ownership semantics so that we can release them properly.

Using the clone() name for the API might be a bit confusing because the above behavior wouldn't be obvious to users, plus there are other ambiguities (e.g. would overridden parameter values get cloned too?).

Perhaps a better way to expose this functionality would be to add a createAssetInstances(...) method to AssetLoader which would allow users to create a number of instances. This would avoid confusion about parameter values and "clone" terminology.

[romainguy]

You only need to clone Material instances if you want to modify the materials of the clones. Otherwise they can be shared too.

[romainguy]

Also @prideout clone() has a specific meaning in Java, we should avoid using that name.

[MingweiSamuel]

So I can see FilamentAsset keeps its cgltf_data* srcAsset at least until releaseSourceAsset() is called. So it would be relatively simple to pass that to AssetLoader.createAsset(srcAsset) to create copies. However AssetLoader also stores a lot of temporary state which gets cleared on each createAsset() call:
https://github.com/google/filament/blob/58bb1f61e6b1ce0fa44fa5eaa101e4fbafe997e0/libs/gltfio/src/AssetLoader.cpp#L164-L167

If mMatInstanceCache and mMeshCache were instead stored in FilamentAsset alongside mSrcAsset, and there was also a new third mTextureCache then you'd have everything you need recreate assets using those caches.

Then there could be a function AssetLoader.createAssetFromAsset(FilamentAsset) as @romainguy suggested in #2344 which returns a new "cloned" FilamentAsset. The method would fail if releaseSourceAsset() had already been called on the input asset.

This way it doesn't require any new loading/entity creation logic. But we would need to change some things in FilamentAsset:

• There are now mSrcAsset, the moved mMeshCache & mMatInstanceCache, and the new mTextureCache as the original source assets.
  
  
  
  • We can have createAssetFromAsset(...) only work on the original copy. So the clone(s) don't store mSrcAsset & caches, as if releaseSourceAsset() had been called. This means we don't have to mess with the mSourceAssetRefCount logic used by ResourceLoader.cpp.
  
  
• These guys need shared_ptr. I think only one ptr for everything, put them in a new struct?
  https://github.com/google/filament/blob/58bb1f61e6b1ce0fa44fa5eaa101e4fbafe997e0/libs/gltfio/src/FFilamentAsset.h#L228-L231
• I _don't_ think they are needed on the transient source data since pointers in the previous bullet keep these alive, but I'm not confident.

Ok that was a wall of text hopefully makes sense.

[prideout]

I have this implemented locally (sneak peek in 72fe00d) but I need to do more testing, add Java / JavaScript bindings, etc.

The API is quite different from earlier proposals but it is simple and works quite well. For example, it does not use shared_ptr in its implementation.

[MingweiSamuel]

@prideout Great stuff, thanks so much!

[davidwarford]

Hay sorry to comment on such an old issue, But i came across this issue while trying to create an array of meshes and i just can't figure it out. and i don't want to make a new issue. @prideout

[romainguy]

If you are using gltfio, we recently added an API to create multiple instance of the asset. See https://github.com/google/filament/pull/3225

[prideout]

@davidwarford gltfio has had support for software instancing since May and you should be able to use createInstancedAsset from JavaScript although I think our only demo for this is in C++ (gltf_instances.cpp).

The JS version of createInstancedAsset takes a pre-sized array for its second argument, and fills it in with instance objects.