Three.js: Blender exporter is very slow

[cmeeren]

I'm trying to convert a Collada file to three.js JSON using your Blender exporter. Conversion seems to be awfully slow. For example, I have a ~50MB Collada file that takes 2–3 hours to convert. Seems a bit much, doesn't it?

It has been a while since I have looked at the console window while converting, but from what I remember, it's the "last phase" where nothing is printed to the console that takes most of the time.

Here's the export script that I'm running (blender_dae2json.py):

"""
Collada to three.js (.dae to .json)
usage: blender.exe --background --python blender_dae2json.py -- in.dae out.json
"""
import sys
import bpy # Blender Python API

# enable Three.js exporter
bpy.ops.wm.addon_enable(module='io_three')

# Get filename arguments
args = sys.argv[sys.argv.index('--')+1:]
fdae = args[0]
fthree = args[1]

# Delete the default objects
bpy.ops.object.select_all(action='SELECT')
bpy.ops.object.delete()

# Import Collada file
bpy.ops.wm.collada_import(filepath=fdae)

# Export three.js file
bpy.ops.export.three(
    filepath=fthree,
    option_export_scene=True,
    option_materials=True,
    option_indent=False,
    )

[mrdoob]

Can you share the 50MB file?

[cmeeren]

Not really. I'll see if I can find a file I'm allowed to share.

[cmeeren]

I now have a file that takes a while to convert. I've uploaded it to WeTransfer, it's available for 7 days at https://we.tl/Xbgffad7ro

The conversion script is provided in my original comment, where you can also see the command I used to invoke the conversion.

When running the command, the three.js exporter will output Writing node ... messages for a few seconds, and then there is no more output for 35 minutes until the conversion completes. That seems awfully long just to convert a 90MB file between Collada and three.js.

[cmeeren]

@mrdoob Did you download the example file?

[mrdoob]

Sorry, I didn't download in time. Do you mind sharing it again?

[cmeeren]

Sorry, it was a very temporary file and now I have no idea which one it was and how I got it. I'll see if I can find it again, or find another, but I probably won't be able to prioritize that for a while.

[manthrax]

Make sure in the exporter settings that you have logging verbosity set to disabled.

The logging slows things Waaay down. and in fact, the way the logging is implemented, actually impacts export times even when disabled, since the construction of logging strings still occurs, even when set to disabled.

If you are comfortable with python, and you need really fast exporting, considering stripping the logging code from the exporter yourself as a one off.

HTH

[cmeeren]

Thank you. I don't think the impact will be that large, though. As I said earlier, almost all of the time taken converting is after the logging is done (a few seconds logging, and then over 30 minutes with no logging output).

[manthrax]

If you want to repost your DAE i can look into it for you a bit further.

I see you're using a script called "blender_dae2json.py"... I am unfamiliar with this script.

Are you importing the dae into blender, using the standard blender ->import ->collada?

And then exporting using the blender->export->three.js(json)?

And if so, at which stage is this taking 30 minutes? How many vertices is your mesh, and is the mesh animated?

If they are animated, are there multiple animated meshes in the file?

In addition to the logging overhead issue I mentioned, there is also an issue where each mesh in the blender scene getting exported, will also export with its own complete copy of all the animations in the file. This is a known issue, and you may be able to work around it. But let me know the details of your usage?

[manthrax]

Possibly related to: https://github.com/mrdoob/three.js/issues/11395

[cmeeren]

If you want to repost your DAE i can look into it for you a bit further.

I'll see if I can find it.

I see you're using a script called "blender_dae2json.py"... I am unfamiliar with this script.

I reproduced it at the top of this thread.

Are you importing the dae into blender, using the standard blender ->import ->collada?
And then exporting using the blender->export->three.js(json)?
And if so, at which stage is this taking 30 minutes?

I'm running the script as described in the OP.

How many vertices is your mesh, and is the mesh animated?
If they are animated, are there multiple animated meshes in the file?
In addition to the logging overhead issue I mentioned, there is also an issue where each mesh in the blender scene getting exported, will also export with its own complete copy of all the animations in the file. This is a known issue, and you may be able to work around it. But let me know the details of your usage?

No aminations.

[cmeeren]

I can't find the original, but I've found two other files. Download here (available for 7 days).

One file is just 7 MB. On my machine, the exporter spits out log messages for 4 seconds, and then nothing is output until it finishes after a total of 1 minute 20 seconds. This is of course not terribly long in itself, but a processing time of over a minute for a 7 MB file seems a bit much, and the exporter seems to scale horribly for larger input sizes. It may help in debugging for quicker turnaround times.

The other Collada file is 267 MB. On my machine, the exporter outputs log messages for 40 seconds. I canceled it after that; I expect it would take the better part of a day to finish, if not more (significantly longer than a linear extrapolation of the small file, which would put it at a bit less than an hour).

[mrdoob]

Could you upload the files in a service that doesn't expire? Maybe https://neocities.org/?

[cmeeren]

I don't want to register anywhere just to upload these files, but you are welcome to do whatever you want with them (they're simply Collada versions of a couple of the files from this site, converted using IfcOpenShell).

[manthrax]

I've downloaded and attempted to export the files you posted.
I see what you mean. The large file contains about 5700 objects, and about 3.8 million triangles altogether.
By watching the logging output, it seems like a lot of the export time comes from triangulating each object.. applying edge split modifiers, and recomputing normals.
Every object in the scene contains unique vertices. There is no instancing, even though the objects appear to have been instanced at one time.. so if you need to move forward with meshes like this, I'd look into getting the models before the hierarchy has been flattened, or do a preprocess to re-combine the instanced parts.

So I agree with your assessment that exporting this is too slow to be practical, but there are steps that could be taken to make it more amenable to fast exporting... and those steps will also translate to more efficient and higher performance rendering, since there wont be 57000 unique geometries being rendered.

[cmeeren]

Thank you for the helpful comment. I expect what you suggest is possible, since the models originate from an IFC file. A couple of questions:

1. What is instancing? Is it different object sharing the same geometry, just with different translation/rotation/scaling?

2. What makes you say the objects appear to have been instanced?

3. Ideally I'd like to write a tool myself that goes directly from IFC to three.js JSON. Where can I read a thorough description of the three.js JSON format?

[mrdoob]

Please, use discourse or stackoverflow for help.