Three.js: e-2-e testing improvements?

[DefinitelyMaybe]

So on my local machine I timed the e-2-e testing process... about 15 minutes 25 seconds.

There are currently 360 examples that this process works through and each test is about comparing two 400 by 250 pixel arrays.

There's just something niggling at me, saying that's pretty slow.

It looks like the current methodology is to loop through each example in sequence. I haven't even looked into the algorithm behind the compare.

Anyone got any ideas on possible improvements?

[donmccurdy]

That's 2-3 seconds per page, which doesn't seem totally surprising for opening pages in an instrumented browser. The built-in timeout is 2 seconds, multiplied by some factor of the page size. Might be able to bring it down if there's some way to hook into a callback when the page is _actually_ done loading, rather than using a timeout?

https://github.com/mrdoob/three.js/blob/bf431973544aa899d892263fa087d2faa4bf246a/test/e2e/puppeteer.js#L192

[munsocket]

Hello. When I realized that CI has 4 separate machines and it always will be faster in x4 I am focused only on CI optimization.

time=16:22 | with rAF hook
time=4:26 | with render promise and parallelism (CI)

I thought that testing script already too complicated and I don't wanted to add another parallelism inside each machine.

But you right. Here the way how to improve page rendering in one machine. Maybe you can improve rendering in x3. Maybe it's is possible because most of the time CPU is waiting.

[mrdoob]

I never run this locally, I let Github Actions do its thing and I see the results in the PR.

[munsocket]

I checked a new approach with bind waiting.

New approach is faster a little bit and code become simpler but it give a new artifacts in screenshots:

webgl_lines_sphere
webgl_loader_gcode
webgl_loader_imagebitmap
webgl_materials_blending_custom
webgl_materials_curvature

Progressive attempts works so well and correct. Difference not so much 15 min vs 12 min.

The only thing that can decrease time is parallel pages because bottleneck right now on CPU. As I know each page in Chrome works in new cores.

[DefinitelyMaybe]

But you right. Here the way how to improve page rendering in one machine. Maybe you can improve rendering in x3. Maybe it's is possible because most of the time CPU is waiting.

This was the kind of thing I was thinking about.

I never run this locally, I let Github Actions do its thing and I see the results in the PR.

I didn't realize that was the intended workflow. May look further into this at a later date but closing for now.