Ar.js: Smoothing over jitter and jerkiness

[janpio]

When you hold the marker and/or phone in your hand, you often have a "jittery" effect on the objects even when you try to hold perfectly still. If you now move one of both, it also adds a "jerkiness" to the movement. Both is more visible with actual models, especially if they are in multiple "depth" levels you get a bit of a parallax effect. The bigger the object, the more pronounced it gets.

While researching a workaround for this, I discovered this StackOverflow answer that offers a way to "smooth" the position and rotation of the drawn objects: https://stackoverflow.com/questions/53380400/aframe-smoothing-position-and-rotation The linked Glitch example uses the Hiro marker, so it is pretty easy to test and play with.

It basically smooths over all the tiny, small position changes of the marker vs. the objects and makes sure the don't directly cause quick movement. If you just move the marker back to the original position quickly, nothing happens to your objects (= no jitter). And even if the object has to adjust its position, it isn't instantaneous but takes some time (= no jerkiness).

I created a fork of the Glitch where I tried to comment the code a bit, also adding a few more debug objects to understand what is going on:

https://glitch.com/~aframe-lerpsmooth

On the marker you have a white box, that is copied onto a red box with the smoothing effect. Move the marker or phone around a bit and you will see that the red box lags behind the white one a bit (which can be minimized by changing the 0.1 to a higher value in the code).
The blue box on the right is a control so you can actually see the jitter and jerk you would normally have.

Unfortunately the code has 2 bugs:

1. The initial position of the "smoothed" box doesn't take the initial position and rotation values into account - so if I would set the #smoothed-box to position="0 0.5 0" so that it isn't half buried, the smoothed box still would be (the original Glitch actually has that problem).
2. Sometimes, especially when rotating the marker fast, the smoothed box "flips" and then animates back to the correct position. I am not sure what that is about and what is causing it.

Any ideas how to fix this?
Any other thoughts on this?

Are there any other approaches to solve these problems that I missed?

[andreyrd]

I believe something similar to this has already been implemented with https://github.com/jeromeetienne/AR.js/pull/422
It originated from #263 and is related to #311
Is this the same issue you're trying to solve?

There's still many improvements that need to be made to it (thinking of allowing for configuration via a component).

It should be available to try out in the dev branch.

[andreyrd]

478 now improves this by allowing configuration of the smoothing via parameters

[nicolocarpignoli]

just for curiosity, does it work properly? in particular, for a-video elements, that showed a lot of jerkiness?

[FelixStumvoll]

Is there any progress to this issue,
Im currently experimenting with different smoothing parameters, but planes are still jerking around.
Especially on mobile devices

[andreyrd]

Are you able to upload a video? I'm able to get pretty good results with just the default settings.

[lkkkeith]

just for curiosity, does it work properly? in particular, for a-video elements, that showed a lot of jerkiness?

Works fine if the video is scaled in sync and not being re-positioned. I tested with iPad Pro.

[nicolocarpignoli]

If bug still persist, please re-open at: https://github.com/AR-js-org/AR.js/

[GBR-422777]

Any update on this, I am facing the same issue

[kalwalt]

HI @GBR-422777 if you have problem, open an issue on the new repository of AR.js: https://github.com/AR-js-org/AR.js/
Please open issues only if your request is a feature request OR a bug.