Three.js: Enhance THREE.Skeleton to act as a "Blender Armature"

[tranek]

I propose that the THREE.Skeleton be enhanced to act like an armature in Blender or a skeleton in 3DS Max/Maya. This would mean that the Skeleton holds bones and animation data like THREE.SkinnedMesh but without any geometry. Implementing this would allow other THREE.SkinnedMeshes to constrain their bones' transformations to the Skeleton's bones (of the same bone name) and use the Skeleton's animations. This is useful for when you have a model of a character and other models that attach to him (his Skeleton) like armor and weapons (which may only have a subset of the bones of the entire Skeleton).

Currently, I have to store the animation in every THREE.SkinnedMesh. Ideally, the animation would be stored into only the THREE.Skeleton object and the other meshes can just attach to it and share its animations. Not only would this greatly reduce file sizes (which helps a lot on super bandwidth heavy model viewer websites like mine), but it also opens the door up for things like ajaxing a new animation directly into the Skeleton at runtime.

I also want to eventually attach particle emitters to bones, but the only reliable way to know that a bone will exist is if it is apart of the Skeleton and not rely on the same bone always being there in meshes that get swapped out a lot.

This new functionality should not change the way that THREE.SkinnedMeshes currently work. SkinnedMeshes should still be able to have optional animations attached to them and don't have to rely on a THREE.Skeleton.

This was mentioned as the last bullet point of #3510.

I originally posted this on Stackoverflow http://stackoverflow.com/questions/25073534/three-js-use-skeleton-like-an-armature-in-blender a couple of weeks ago, but didn't get any replies so I'm going to assume it's not implemented/possible yet, meaning it is ripe for a feature request.

[mrdoob]

@ikerr thoughts?

[ikerr]

Hi @tranek ,

I think the current implementation will do what you're looking for: apply a single animation to a single skeleton and use that skeleton to drive multiple meshes. You'll notice that THREE.SkinnedMesh has a bind() function that takes a skeleton and an optional bind matrix - you can use this to attach a single skeleton to multiple meshes. For example:

var skeleton = new THREE.Skeleton( bones );
var meshA = new THREE.SkinnedMesh( geometryA, material );
var meshB = new THREE.SkinnedMesh( geometryB, material );

meshA.bind( skeleton );
meshB.bind( skeleton );

where bones is an array of THREE.Object3D instances that exist in your scenegraph.

Note: since the skeleton is shared, you only need a single animation to animate both meshes.

Hope that helps. Let me know if you have any questions/comments.

[tranek]

Hi @ikerr ,

Thanks for the response. I'm not entirely sure that I understand how to implement this. Using a Blender workflow:

1) I would load in an animation into Blender so that there's only an armature in the scene
2) Export the armature to .js only including bones (no geometry or animations since THREE.Skeleton doesn't store either)
3) Import into my three.js app by using the standard JSON loader and in the handler use the bones as the parameter to initialize a new THREE.Skeleton
4) Export from Blender meshA and meshB to .js with just geometry, bones, and skinning but no animations (or just one of them with the animation?).
5) Import meshA and meshB into my three.js app using the standard JSON loader and in the handler use their bind() methods on the THREE.Skeleton that I created in step 3.

Where would I import/add the animations? I read your post as if meshA or meshB would have the animation (but not both meshes) and the other would share it. But how would this work if both meshes only have a subset of the bones of the original THREE.Skeleton? If they don't both have all of the same bones, then whichever one has the animations embedded in them wouldn't have the animation info for the missing bones (if I understand how that part works correctly).

If I understand your suggestion correctly, it would seem that I would need to create a "dummy" THREE.SkinnedMesh with all of the bones but no geometry (can I make one with no geometry?) that would hold all animation info.

I hope that my confusion makes sense. Thanks!!

[ikerr]

Hi @tranek ,

I'm not sure if THREE.JSONLoader can do everything you want - we use custom loaders. I can give you an idea of what we do, and maybe that will help...

Our animations are stored in separate files and are quite simple: a list of channels with keyframe data. The channel names can be mapped to bones in the scene. For example, an animation file might look like this:

{
  "LeftLeg": {
    "position": { 0, 0, 0, 0, 1, 0, 0, 1, ... }
    "rotation": { 0, 0, 0, 0, 1, 1, 0, 0, 1, 1, ... }
    "scale": { 0, 1, 1, 1, ... }
  }
  ...
}

Our animation files have a flat structure - the channels are not hierarchical. This allows us to “bind” the animations to any hierarchy at runtime by matching channel names to bone names. For example, you can do something like this:

var animation = createAnimation(root, channels);

where root is the root of your THREE.Object3D hierarchy, channels is your channel data and animation is the resulting THREE.Animation. Relating this to your first post, if you have a character with interchangeable parts, you can have a single set of channel data and bind that to multiple object hierarchies (e.g., weapons, armour, etc.).

To sum things up, here are a couple points:

• We don’t use THREE.JSONLoader and don’t embed animation data in models - we load animations separately.
• If your animation data is stored separately, you can bind animation channels to arbitrary object hierarchies at runtime.

[tranek]

Hi @ikerr,

I've been trying every combination of code and not getting very far. Using the bind() method of the meshes, I can get meshes to share the animations and they animate 'correctly' but are in the wrong positions/rotations.

My meshes are weird. They are all positioned at the origin and seem to have some extra transforms applied to them before the animation takes over. For example, when I embed the animation into the meshes directly, the only way that they are in the correct position/rotation (in a THREE.SkinnedMesh) is if I follow a weird procedure in Blender when exporting them. I have to save out the armature + animation as a separate .blend file and then use that with the link function on each item to that .blend file and then constrain all of the bones' transforms in the mesh to the armature's bones's transforms.

I'm not sure exactly what happens different when I use the link function in Blender, but some magic happens there that applies the correct transform to the mesh to match it up correctly with the armature in three.js. I don't know how to replicate what the Blender linking is doing since I don't know what it is doing heh.

Directly copying the skeleton (a la bind()) doesn't seem to work since the correctly animating THREE.SkinnedMeshes have bones that are in different positions/rotations than the base Skeleton's bones.

If that all sounds confusing, then that's okay because I have a headache from trying to explain it. Anyways, I'm starting to give up hope since I have such a weird situation. I don't have access to the artists to fix the models -- they are as is.

I'm just going to have to save out every animation for every item separately and then AJAX them in to every item as necessary. Just extra harddrive space and bandwidth that I was hoping to save on. :(

Thank you for trying to help!

[ikerr]

Hi @tranek ,

THREE.SkinnedMesh's bind() function takes a skeleton and optionally a bind matrix, which will allow you to correctly line up your skeleton and mesh. For example, if your mesh is offset from your skeleton by 2 units along the x-axis, you can pass in a bind matrix to negate that offset.

[tranek]

Hi @ikerr ,

That makes sense, but how do I calculate the bind matrix? While looking at correct THREE.SkinnedMeshes (ones that animate in the correct position/rotation because the animation was loaded in with them instead of trying to bind() to a skeleton), their bind matrices are all the identity. Do I have to go through and manually lineup all the meshes and calculate the bind matrix from that? I hope that is not the case because I have a lot of meshes and it just wouldn't be feasible to manually calculate the bind matrix for all of them by hand.

Thanks for your patience with my inexperience with this :)

[ikerr]

Hi @tranek,

There should be some way to infer the correct bind matrices from your data, but I don't have access to your data so I can't really tell you how to calculate them :) If you can set up a jsfiddle, I might be able to have a look and point you in the right direction.

--Ian

[tranek]

Hi @ikerr ,

Sorry for the delay. Reddit discovered my website over the weekend and I've been dealing with the fallout from the "Reddit Hug of Death" and the thousands of eyes looking at it and reporting bugs.

What would you need in the jsfiddle? I am assuming that you would need the main skeleton with the animation (I've actually have been just using a "main" mesh of sorts that has all of the bones and some geometry that stays static/doesn't swap out like the other items and binding to its skeleton instead of making a standalone THREE.Skeleton if that makes sense) and a couple of models with correctly embedded animations and the same models without embedded animations?

I've honestly never used jsfiddle before, I assume that it can handle loading .js files from external sites (the models in this case)? Or would I need to enable some sort of CORS flag somewhere on my server to make sure jsfiddle can access my models?

Thanks!

[mikebourbeauart]

I think I'm trying to accomplish something similar to the original post here.

Say I have a single skinned mesh file called skelmesh.fbx without any animations on it- just a skinned mesh and bones. I also have several animation files called idle.fbx, walk.fbx, and run.fbx containing only bone and/or blend shape animation data. I'd like to use these animation files to drive the skinned mesh file. This is a similar setup to UE4 and Unity (at least with bone animations, not sure if blend shape animations require a mesh in the file) where you have a skeletal mesh being driven by joint animation data.

Is this setup possible and if so how should I go about it? Is the bind method the key here?

[mikebourbeauart]

figured it out here: https://discourse.threejs.org/t/fbx-drive-a-single-skinnedmesh-geometry-bound-to-skeleton-in-maya-with-multiple-animated-skeletons/4072

[donmccurdy]

I think this issue can be closed as "obsolete" — it predates the new animation system, and animations can now be serialized separately from models, and reused with different objects by assigning a different root to AnimationMixer instances. For example:

var walk; // THREE.AnimationClip

var mixer1 = new THREE.AnimationMixer( mesh1 );
var walkAction1 = mixer1.clipAction( walk );
walkAction1.play();

var mixer2 = new THREE.AnimationMixer( mesh2 );
var walkAction2 = mixer2.clipAction( walk );
walkAction2.play();