Godot: Add built-in fixed timestep interpolation

[lawnjelly]

Following discussion here #10388 I've been working on potential enhancement to the engine.

Issue description:

There is as yet no support for fixed timestep interpolation, you have to implement it yourself.
https://www.gamedev.net/blogs/entry/2265460-fixing-your-timestep-and-evaluating-godot/

The past few games I've made I've written some hacky interpolation in gdscript, however now I'm tinkering with the source I've managed to make a module with a smoothing node type that will do timestep interpolation for you, without any code:

https://www.youtube.com/watch?v=SFLwCR2KEJ8

After spending a few days on it, 3 avenues seem to be interesting options:

1. Build optional interpolation ability directly into nodes (perhaps spatial?)
2. Add a smoothing node to the main engine
3. Add better support to allow use of smoothing nodes in modules, or smoothing within gdscript.

_More on option 3_

Interpolation is already possible to an extent but really needs the main_timer_sync to expose to gdscript and modules the interpolation fraction, the fraction through the current physics tick that the current frame is being rendered at. As far as I can see that is not available at the present time, I have modified the timer to allow this for my module.

At present you can for instance make a note of the OS time from gdscript at the last physics tick, and the OS time at the rendered frame, and calculate the time difference, but this is subject to jitter because you actually need to know the logical time that the physics tick would take place, rather than the actual time it is calculated.

Anyway I'm thinking of starting by doing a small PR to allow give access to the interpolation fraction in gdscript and modules. But would be interested to gauge the support for some kind of inclusion of ability to do interpolation as a first class feature of Godot.

[Calinou]

I wrote a demo that relies on Node2D.set_as_toplevel() here: https://github.com/Calinou/godot-physics-interpolation-demo

As someone who owns a 144 Hz display, I would really appreciate having this built-in. However, we need a way to prevent interpolation from being performed when needed. This is needed when teleporting objects, for instance, so that they don't appear to interpolate through other solid objects. I can see several ways to do this:

• Add a property to PhysicsBody that can be toggled when needed. Can be tedious, but it's flexible.
• Add a parameter to the KinematicBody/RigidBody movement functions. Can be tedious, but it's flexible.
• Add an user-defined "maximal interpolation speed" property to PhysicsBody which defines the maximum speed at which the object is reasonably expected to travel in one second. If this speed is exceeded in a single frame, no interpolation will be performed. This still requires a bit of manual setup, but it's much more fire-and-forget. Still, if the maximal speed is set too low, interpolation will stop working when the object moves really fast.

[lawnjelly]

The demo looks good to me, although I believe it may suffer from the same problem that the engine doesn't currently support : the start time at integrate forces is the actual time the tick is calculated, not the the logical time of the tick.

For example, at a tick rate of 1tps, if the current frame time is 1.0 second, and the physics tick was due at 0.5 seconds, but is processed late (just before the frame), as far as you are concerned the physics tick started at 1.0 second, and the frame starts at 1.0 second, and the interpolation fraction is 0.0, whereas in reality the interpolation fraction at the frame should be 0.5. In fact in your code the delta passed to _process is the time since the last frame, not since the last logical physics tick. So that might be another source of error - but as I say it may currently be impossible to get the right value.

This is why a slight modification to the timer code in Godot main engine may be necessary to expose this and deal with this issue, which can be a source of jitter. I will have to have a proper look, at the moment I've completely hacked apart main_timer_sync and replaced it with a simpler version because I didn't want to deal with the jitter fix while debugging. It is probably possible to turn off the jitter fix by setting it to 0.0 but I need to confirm this empirically. In fact with proper interpolation, I don't think there is need for the jitter fix.

Agree that interpolation should be essentially free when turned off, both in terms of processing, and preferably in terms of memory use too. Having a separate smoothing node achieves this (method 2), whereas making it part of the node itself (method 1) requires some care. However ultimately method 1 would be easiest to use, especially for beginners.

Adding a property to physics node sounds viable, but you might also want to use it for non physics stuff like cameras so there is an argument for adding it to spatial. You could do it pretty cheaply just by having a pointer to an interpolation property that is either present or not on the node. The bigger problem is getting such a use of interpolation without a proxy to work without throwing out the physics (or other things). At the moment, if your interpolation code moves e.g. a rigid body translate and rotate to smooth it, it also moves the physics, giving a feedback loop. I had a little go at this and trying to prevent the transform change being propagated but opted for the separate node method for now. Really you would ideally have accessible both the transform at the physics tick (for physics, AI etc) and the interpolated position for rendering etc.

Interesting idea about the maximum interpolation speed / distance. I've always simply had a function which gets called explicitly, called teleport, which both moves the object but also sets both the previous and current values for transform. This is the one 'cost' of interpolation, you have to remember to call teleport rather than e.g. set the transform directly. But in practice I've never had a problem with this.

[fracteed]

@lawnjelly sounds very promising! I have struggled with jitter issues over the years with Godot. For my main game, I currently have resolved most of them and it is pretty smooth. I find that Godot will mostly give you a solid jitter-free experience as long as I can comfortably stay over 60fps and use vsync (either in engine or in the GPU settings).

However jitter/judder appears pretty much as soon as it dips below 60 fps with vsync on. If I have no vsync on, even if I am well above 100 fps, it is not a smooth experience (not even factoring in screen tearing). Would be interested to know how well your solution handles in these situations. Can't say I have noticed any improvement when the jitterfix option was added to Godot.

[girng]

if this fixes a player who has a 144hz monitor playing a godot game that coded their movement in physics_process (defaults to 60fps), instead of _process; then i think it will be a great addition to the engine. good work, lawnjelly

[lawnjelly]

A little update on this .. I have put in the first PR for adding the interpolation fraction. I did some testing with the existing main_timer_sync code, which includes the jitter fix and is a little hard to fathom lol :smile: but empirically with jitter fix set to 0.0 it produces pretty much the same results as my simplified main_timer_sync (there is some small floating point variation (+- 0.001 or so), but nothing serious I think).

I've also written a simple (for now) delta time smoother to work alongside the main_timer_sync. This is because there are 2 major sources for jitter, one is the need for timestep interpolation, and the other is the lack of correct timing information coming from the OS. I will write a separate issue for this later to discuss.

Still no joy on working out a good place to attach the interpolation to the nodes themselves. It has occurred to me if we keep the usual transform as holding the current physics transform, the interpolated transform can perhaps be used only when propagating to the children. However I haven't quite worked out where the parent -> child transforms are getting concatenated yet.

[lawnjelly]

Further update.
I now have a proof of concept working with interpolation built in to the spatial node, such that there is a bool you can turn on in the editor for anything derived from spatial to enable interpolation.

I tracked down the actual concat of the parent and local transform to Spatial::get_global_transform(). It is actually a lazy function, delaying the actual concat until needed and marked by a dirty flag, which makes sense.

I managed to get it working by adding an internal physics notification (for the fixed update) and internal process notification (to mark dirty) for spatial, and banditing the get_global_transform() function, such that the parent retains the uninterpolated transform, but the children get the interpolated version.

However, I still don't have it working with the physics nodes, there is still feedback going on from the interpolated global transform to the physics representation in bullet.

Best solution?
I believe I would be able to get it working with physics, _however_ I am getting concerned that while integrating interpolation into spatial is the easiest solution as far as the user is concerned, it makes the engine code itself overly convoluted, a bit hacky and hard to understand, and prone to bugs in the future.

So at the moment I am in favour of the simplest and most maintainable solution as far as the engine is concerned, which is to have a separate smoothing node, even if it means marginally more work for users. This also guarantees there will be no extra costs in terms of processing and memory use if a game is not using fixed timestep interpolation (there seems to be some interest in alternative methods also, such as semi-fixed timestep and frame delta physics stepping).

Next
So I am now thinking of concentrating effort on refining the smoothing node, and then making it available (as a module and possibly test build of Godot) for testing / feedback.

Note that in any case it will be dependent on the interpolation fraction PR #30226 so will have to wait until this is merged.

[lawnjelly]

Now access to the interpolation fraction has been merged, I have made first versions of both a c++ module and (on suggestion from Calinou) a gdscript addon to add smoothing nodes to allow fixed timestep interpolation, for anyone who is interested in trying this. There may still be a few little bugs to sort and I'll refine them over the coming weeks but they are mostly working fine I think:

GDScript addon:

https://github.com/lawnjelly/smoothing-addon

c++ module:

https://github.com/lawnjelly/godot-smooth

Of course it is also possible to do the interpolation yourself now, so if you want to use these as examples to work from go for it :smile: . Also suggestions for changes / bugs please let me know, I guess via issues on the repositories.

Once the bugs have been worked out I'll have a look at whether it is possible to put the gdscript version in the Godot asset library. It requires a build with the interpolation fraction PR so it might have to wait until 3.2 is released.

Some interesting quirks

Normally for interpolation I use slerping for quaternions, and handle scale separately. As Godot stores rotations and scaling combined in a basis I decided to try out simply lerping the basis as an alternative. To my surprise it actually looks quite reasonable. I suspect there is some contortion going on during the lerping but it is hard to see even at low tick rates, so is unlikely to be a concern at over around 20 ticks per second.

On the other side of the coin decomposing a Basis back to a quaternion doesn't seem to work well with any scaling applied to the Basis (I think the gdscript version flat out refuses to do it). I have left the slerp method available, but it would probably mean you would need to apply any scaling to a parent node.

Now it is actually possible to do the interpolation with the PR I will close this issue after a couple of days just in case of more comments, as it is effectively resolved. :+1:

[semirix]

I wanted to add my two cents here instead of making a new issue despite it being closed.

I don't think having a separate module for this feature is appropriate. I see a two pronged approach to implementing this in a more effective way, albeit idealistically:

1. Going with what was said prior - add a tick-box to PhysicsBody to enable interpolation. Having it on Spatial doesn't make sense because there isn't really a sane use case in my opinion for a Spatial node to need to automatically interpolate. The movable physics bodies are the only nodes that really require this behaviour to be done automatically. If there is a use-case for it, then I withdraw my comment.
2. Add a new overrideable function in PhysicsBody called _timestep_interval for user-defined interpolation. This function would run in the same step as _process but never in the same step as _physics_process which takes 3 arguments:
   
   
   
   • delta: time since last physics frame
   
   
   • next: time to next physics frame (this would be estimated but I'm not sure how feasible/realistic that is)
   
   
   • state: a handle to a shadow PhysicsDirectBodyState object which the interpolation calculations are applied to. This state is always overridden whenever the normal state is written to or set (solving the problem of rubber-banding when teleports are done on raw transforms).
   
   

This would allow users to manually handle the interpolation in a more controlled manner. It might be handy to have a function that can integrate the forces in state and return a new PhysicsDirectBodyState based on the delta and next variables. This approach to interpolation would be the most flexible because the tick-box should work for most users and for those that it doesn't, they have an overrideable function they can work with to fit their specific needs.

I don't know how much of this implementation is realistic but I would appreciate comments and criticisms to better understand the limitations of this approach. Thanks!

[lawnjelly]

Putting interpolation in the physics has always been one of the options.

@AndreaCatania has been investigating this approach here:
https://github.com/godotengine/godot-proposals/issues/671

Having it on Spatial doesn't make sense because there isn't really a sane use case in my opinion for a Spatial node to need to automatically interpolate.

Consider people may be making games without using Godot physics _at all_. I've made several. Also consider things like cameras which may not require physics.

However there is another possibility, when me and Andrea discussed this, one of the ideas was to use a physics dummy object, which has the interpolation but no actual representation in the physics world. This is another way of getting around the problem, while keeping the interpolation in the physics module.

I don't think having a separate module for this feature is appropriate.

There is a good argument for having support for interpolation in core, however the decision is ultimately down to reduz, and he is opposed to this.

[semirix]

@lawnjelly Thank you for taking the time to address my comments. Now that you've mentioned it I think letting people do things their own way rather than someone telling them how to instead makes putting the feature on the Spatial node a better idea overall. That was really my only point in terms of "sane use-case".

[nezvers]

If I'm understanding it right, that this is trying to solve situations where _phisics_proces goes out of sync with rendering and it appears like a frozen frame with a consecutive jump? In that case, I raise the question - why only Spatial?
I focus mostly on pixel art games and I'm in the process to document some visuals for a bug report because that jump is VERY obvious.
Jitter like that shouldn't be "expected", because it makes it look really bad and begs the question of why there's a separate physics_process update if it makes games look bad. I think such a fix should be there in the first place so any new developer wouldn't drop Godot saying - but GameMaker doesn't have jitter. GameMaker users mostly write their own collision solving and all their physics happens at step callback (like Godot's "_process").

1. Going with what was said prior - add a tick-box to PhysicsBody to enable interpolation. Having it on Spatial doesn't make sense because there isn't really a sane use case in my opinion for a Spatial node to need to automatically interpolate. The movable physics bodies are the only nodes that really require this behavior to be done automatically. If there is a use-case for it, then I withdraw my comment.

IMO that is needed for anything you can move, it might be Node2D carrying Area2D as a child.

[lawnjelly]

I totally agree .. mentioning spatial is only because my initial investigation was in 3D.

If you check out my smoothing addon:
https://github.com/lawnjelly/smoothing-addon

This has 2D as well as 3D smoothing nodes. :+1:

[nezvers]

@lawnjelly Yeah, soon after found out you had it in repository. I'm checking it right now and it's much bigger script than I'd like it to be for such task. More flag flipping than actual interpolation. Non the less it's amazing learning material.

[Calinou]

More flag flipping than actual interpolation. Non the less it's amazing learning material.

The flag flipping is just an abstraction. We could probably remove those abstractions to make the script shorter :slightly_smiling_face:

[lawnjelly]

@lawnjelly Yeah, soon after found out you had it in repository. I'm checking it right now and it's much bigger script than I'd like it to be for such task. More flag flipping than actual interpolation. Non the less it's amazing learning material.

Feel free to use it as a reference to do your own interpolation.

As well as being something users can just plug in to their own project, it is also intended as a demonstration of using Engine::get_physics_interpolation_fraction. :relaxed: