Assimp: Support glTF 2.0 Specification

[dhritzkiv]

It'd be nice to support importing (and exporting) of glTF 2.0 files.

Khronos just released the finalized glTF 2.0 Specification. For the most part, the syntax is changed so now properties are defined as arrays instead of objects/maps, with references defined as indexes instead of keys. The spec also defined PBR-based materials.

[kimkulling]

You are right, this would be great!

[jamesgk]

I started a branch for this:
https://github.com/jamesgk/assimp/tree/gltf2-export

One issue for now is that, rather than duplicating most of the glTF 1 exporter, I modified it to export glTF 2. So on this branch it's no longer possible to export glTF 1. I also have not implemented glTF 2 importing.

[dhritzkiv]

Sweet!

I think it'd be important to still support glTF v1, for legacy reasons, especially on import. I wonder if it'd be easy to sniff the version and then choose the right Parser/Importer.

The default version for glTF export should be v2, but v1 export might be specified via the format option: --format=gltf1

[realvictorprm]

@dhritzkiv is this in a working state?

[dhritzkiv]

Hi @realvictorprm. I'm not working on this feature - I merely suggested it. See @jamesgk's work on glTF 2 support so far. It likely has some work to go before being integrated into assimp

[jamesgk]

Yes, this definitely needs more work. I was just going to push some
fixes to animation exporting but then Daniel's suggestions still need
to be implemented.

[jamesgk]

An update: on a new branch I separated glTF1 and 2 into two (largely duplicated) exporters, so now at least the glTF2 exporter should be mergeable: https://github.com/jamesgk/assimp/tree/gltf2.

[realvictorprm]

Thank you for your effort. I always appreciate open source work even if I'm not working mainly in the repo. If you need any help because you have no time don't hesitate to tell me.

[jamesgk]

@realvictorprm thanks! I probably won't have time in the near future to work on glTF 2 importing. So if there's much demand for that, any help would be welcome.

[jcowles]

FYI, glTF 2.0 seems to be the only future for glTF.

New tools being developed are already starting to favor 2.0, some even lacking 1.0 support entirely.

That said, yes, 1.0 and 2.0 would be ideal.

[dhritzkiv]

Hi @jcowles. glTF2 export functionality is in assimp master, and there's some work from me to add glTF2 imports as well, here: #1423

[pjcozzi]

@dhritzkiv all of us working on the glTF spec are excited to see glTF 2.0 support in assimp! Is there any documentation on the extent of glTF 2.0 supported, roadmap, and anything Khronos can do to help?

[dhritzkiv]

Thanks @pjcozzi!

In terms of assimp's support of glTF 2.0: much of the core parts of the spec are implemented:

• asset
• buffers
  
  
  
  • except GLB - see below
  
  
• bufferViews
• scenes / scene
• nodes and meshes
  
  
  
  • ~[update 2017-09-18] there's a PR fixing primitives output here: #1446~ - merged!
  
  
• images
  
  
  
  • both via external uri and embedded base64-encoded data uris
  
  
  • unsure about referencing via bufferViews. I think it's in there
  
  
• textures
• materials
  
  
  
  • the basic properties, pbrMetallicRoughness and the pbrSpecularGlossiness extension are fully implemented.
  
  
• accessors, sort of - see next section
• I'm unsure about cameras support. I think it's in there.

What remains to be implemented/fixed is this functionality:

• binary_glTF
  
  
  
  • I started work on it here a while ago, but I've not gotten far.
  
  
• animations/joints/skins/morph targets (and by extension, their accessors)
  
  
  
  • assimp currently outputs some data for these, but it's copied from assimp's glTF 1.0 implementation, so it's incorrect
  
  
• sparse accessors

I didn't get around to fixing these, partly due to it not being a priority for my use cases, and partly because I'm unfamiliar with it.

Persisting name properties is a bit inconsistent, and extras properties also should be serialized and persisted.

---

glTF 2.0 extension implementations that are on the road map include:

• KHR_material_common/KHR_materials_cmnBlinnPhong
• lights
• Draco compression (to replace the now removed Open3DGC compression)

I think we'll wait until the extension specs are finalized before implementing these.

Speaking of KHR_material_common: several of us (in #1423) were unclear as to the state of KHR_material_common (or its equivalent) for glTF 2.0, partly due to to scattering of issues/PRs that address it, and the moving-target/ work-in-progress implementations in other loaders (namely three.js). As I understand it, the current plan is to implement a "common material" via KHR_materials_cmnBlinnPhong, yes? Is this the PR that we should be following: KhronosGroup/glTF#1075?

---

As for further help, it'd be appreciated if Khronos was able to verify the correctness of the export functionality (and import too, for that matter) of glTF 2 models. I tested against most of the models from the Khronos sample repo, and most pass the glTF validator with flying colours. The ones with animations/joints/skins/morphTargets present many errors.

If there's any interest from Khronos members in contributing further code and fixes, please do!

[jcowles]

Awesome.

However, testing master with a random glTF 2.0 file I grabbed from SketchFab, I'm getting a segfault with this stack:

[dhritzkiv]

Mind linking that model? I could try tracking the crash down

[jcowles]

Here you go:
https://sketchfab.com/models/dded7949a3db4ba4a09ef3ddbd13f319

[dhritzkiv]

I couldn't reproduce the crash on my end. Could it be platform dependent? I think there were some recently discovered issues with MSVC builds. (In which case, I'll let those with Windows experience address that) It could also be a general memory bug that might happen to surface on your machine.

---

That being said, the model didn't export perfectly: the tires, lights, and undercarriage are missing.

So, good test model (though, I have some simpler ones that might be easier to debug). I suspect that it may to do with having multiple primitives per mesh (a pretty glaring implementation omission on my part, if true).

[jcowles]

Well, it's odd that my glTF 1.0 and OBJ files load fine, but the new glTF 2.0 loader crashes.

Also, given that the MSVC run-time is killing the process, it definitely looks like a memory bug.

[dhritzkiv]

I'll try to take a crack at the memory bug on Monday when I'm at my workstation with better tools

[jcowles]

Alright, if I get inspired, I may take a look tonight too (doubtful though).

[jcowles]

Also, the point at which it crashes is non-deterministic, if I run my script several times, it will crash with different stacks.

[jcowles]

To to fully exclude my code, I tried this with assimp.exe and it also crashes.

[jcowles]

Ok, one last bit of information, the assimp.exe validator crashes on this:

void ValidateDSProcess::Validate( const aiNode* pNode)
{
    if (!pNode)ReportError("A node of the scenegraph is NULL");
    if (pNode != mScene->mRootNode && !pNode->mParent)  // < Crash here on pNode->mParent

Looks like pNode is not equal to mRootNode and it's pointing at some illegal memory location (not null).

[jcowles]

Your oversight comment above is exactly what's happening, in this code, it assumes exactly one primitive, allocates one mesh, but then proceeds to write multiple meshes, corrupting memory:

if (node.mesh) {
    ainode->mNumMeshes = 1;
    ainode->mMeshes = new unsigned int[1];

    int k = 0;
    int idx = node.mesh.GetIndex();

    for (unsigned int j = meshOffsets[idx]; j < meshOffsets[idx + 1]; ++j, ++k) {
        ainode->mMeshes[k] = j;
    }
}

In glTF, primitives are effectively draw calls, so it seems extremely common that meshes will have multiple primitives.

[dhritzkiv]

Got catch.

I've tracked it down to this commit: a0d97505e5a53a9aa3a734d6bd3ad790234e6e09

Reverting it works, however, now nodes reference multiple meshes with one primitive instead of one mesh with multiple primitives. Should be an easy fix, I think, though I don't quite know where to start. Probably in glTF2Exporter::ExportMeshes() or glTF2Exporter::ExportNodeHierarchy(). I'll take another look tomorrow.

[jcowles]

If primitives all have the same material and mode, they could be flattened into a single mesh, I think.

However, since Assimp doesn't support face material bindings, you must also support the code path where the material/pipeline state inst shared between primitives, which means having multiple meshes for a single node. This seems like a good starting point, since it has to be implemented as the fallback anyway.

[jcowles]

Oh hey, the fix is easy:

Should I send a PR?

[dhritzkiv]

@jcowles yes, please!

I've spent the last hour wracking my brain on how to flatten the meshes' primitives into a single mesh with multiple primitives, in code (I'm not too nimble with C++), so I'd love to see, and possibly use your solution.

[jcowles]

There you go!

[jcowles]

Oh, hrm, it's also kind of a bummer that this implementation throws away all the glTF node names.

[pjcozzi]

Wow, @dhritzkiv, thanks for all the details in https://github.com/assimp/assimp/issues/1301#issuecomment-329988412!

Speaking of KHR_material_common: several of us (in #1423) were unclear as to the state of KHR_material_common (or its equivalent) for glTF 2.0, partly due to to scattering of issues/PRs that address it, and the moving-target/ work-in-progress implementations in other loaders (namely three.js). As I understand it, the current plan is to implement a "common material" via KHR_materials_cmnBlinnPhong, yes? Is this the PR that we should be following: KhronosGroup/glTF#1075?

The plan is still to separate out lights and materials from the glTF 1.0 KHR_material_common extension.

As for the tweaks in https://github.com/KhronosGroup/glTF/pull/1075#issuecomment-328408375 and renaming KHR_materials_cmnBlinnPhong to KHR_materials_common, I do not know if this was confirmed.

@donmccurdy @UX3D-nopper do you know?

As for further help, it'd be appreciated if Khronos was able to verify the correctness of the export functionality (and import too, for that matter) of glTF 2 models.

I bet we can find some help. I will ask around!

If there's any interest from Khronos members in contributing further code and fixes, please do!

Nice, will also ask around!

[donmccurdy]

as for ... renaming KHR_materials_cmnBlinnPhong to KHR_materials_common, I do not know if this was confirmed.

Not sure yet; I would hold off on implementing common materials a bit. Feedback on the issue is very welcome, though!

[krupitskas]

Where can I try assimp gltf 2.0 importer? Because now I have error that no importer found for this format, where format is .gltf

[dhritzkiv]

Are you using recent code from master on Github? The releases don’t include gltf 2 importing.

On Nov 24, 2017, at 07:02, Nikita Krupitskas [e-mail] wrote:

Where can I try assimp gltf 2.0 importer? Because now I have error that no importer found for this format, where format is .gltf

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or mute the thread.

[krupitskas]

Yeah, Im fighting with this issue right now https://github.com/assimp/assimp/issues/1566
Thats why Im tried libassimp-dev and gltf not imported.
I just downloaded assimp and build it from sources, checked in asssimp_qt_viewer and everything works.
Do you know how to solve that problem?
Thanks

[mario-132]

Has it been released as a official release yet?

[m0nologuer]

This seems like a great feature! What can we do to make it happen!

[dhritzkiv]

@Tim4135 @m0nologuer glTF importing and exporting is part of at least the 4.1.0 release. In addition, there have been more features and improvements to glTF2 features in the many commits since.

[EvilPudding]

Does external binary import glTF2 work? Do animations on glTF2 work at all right now?

[singhraj82]

Is glb for glTF2.0 supported in current assimp version? Or is it expected to be supported in 5.0 release ?