Godot-proposals: Add an absolute time (DSP time) feature

[rezgi]

Describe the project you are working on:
A rhythm game where all events and animations are synced to MIDI data

Describe the problem or limitation you are having in your project:
I struggle to track and trigger events based on musical data (tempo). Since I can only execute logic in _process() and _physics_process(), I depend on the framerate while I need to trigger an event at exactly 2 seconds for example. But it's never 2s precisely and I constantly compensate the delay when the tempo (bar or beat) happens between 2 frames.

Other engines have APIs like dspTime (Unity) or TimeSynth (Unreal) to manage that and I'm wondering if it's achievable in Godot. I don't know about the complexity of implementation, from what I understand these precise timing methods seem to run on their own thread I guess ?

Describe the feature / enhancement and how it helps to overcome the problem or limitation:
By having access to an API that can trigger events on precise timings like 2s (00:02:0000) which will allow easier programming for musical mechanics. Game logic will still run on main thread but at least we'll have another parallel logic that we can rely on to track the progression of the tempo for example.

Describe how your proposal will work, with code, pseudocode, mockups, and/or diagrams:
if get_absolute_time() == 2 print("Bar 2 reached")

If this enhancement will not be used often, can it be worked around with a few lines of script?:
There's no way to get absolute time and we need to check for approximations, which is ok for music playback and simplistic rhythm games but becomes hard when we want to deeply sync rhythm and game mechanics.

Is there a reason why this should be core and not an add-on in the asset library?:
I think this kind of implementation really depends on the engine low-level process and I guess it needs an alternative thread to run this implementation. Maybe a C++ plugin could do that ? I'm not advanced enough to know.

[Jummit]

There is OS.get_ticks_msec(), if that works for you.

[Calinou]

Related to https://github.com/godotengine/godot/issues/32382.

[rezgi]

There is OS.get_ticks_msec(), if that works for you.

Indeed but where do I call it and check it ? We can run logic only in process() functions so we stay dependent on delta duration, which means that when I'll check if OS.get_ticks_msec() equals 2s for example, in _process(), time may already be 2.001 for example.

Related to godotengine/godot#32382.

Indeed this is quite related. It's about finding a way to trigger events at an absolute given time even though the game runs on an irregular tick (delta). Unity provides dspTime and we have to manually code the logic to preload sounds when it gets close to this absolute time for example. And Unreal provides TimeSynth that tackles this issue more precisely and out of the box.

I guess using OS.get_ticks_msec() and anticipate it should be enough, having events trigger on absolute time maybe isn't the right approach for a game I guess ?

[starry-abyss]

Isn't it already available?

https://github.com/godotengine/godot-demo-projects/tree/master/audio/bpm_sync
https://www.youtube.com/watch?v=kXG1jXc8k2k

[kerskuchen]

I think @starry-abyss is correct in that what @rezgi wants to do is already available (you can look at https://docs.godotengine.org/de/stable/tutorials/audio/sync_with_audio.html for the corresponding tutorial).

The problem is though, that as soon as @rezgi decides to play a sound at an exact moment in time (i.e. at a specific beat), it may be played out of sync by up to n milliseconds (where n is the audio buffer size??) and it may sound awful. Here is a GDC talk about this exact problem https://gdcvault.com/play/1017877/The-Audio-Callback-for-Audio.

The solution for this would be a new API like Unity's AudioSource.PlaySheduled. @rezgi you may get away with https://github.com/godotengine/godot/issues/32382#issuecomment-632691623 from the related issue @Calinou mentioned.

[rezgi]

Thanks @kerskuchen for the info. As I was working on a DAW-like metronome in Godot, I found it was quite constraining to depend on process() to check for the exact timing, so I was thinking of APIs like Unity's PlayScheduled or Unreal's TimeSynth that try to solve this specific problem.

Seems like the other issues have pointed that problem too, as I'm not technically advanced in the matter, I don't know how this feature could be implemented aside from having another thread only for audio.

[benjarmstrong]

I'm also working on a music/rhythm game with events synced to MIDI data and have experienced the same problem.

There is OS.get_ticks_msec(), if that works for you.

This is exactly what I use, but alone it wasn't enough, because...

The problem is though, that as soon as @rezgi decides to play a sound at an exact moment in time (i.e. at a specific beat), it may be played out of sync by up to n milliseconds (where n is the audio buffer size??) and it may sound awful. Here is a GDC talk about this exact problem https://gdcvault.com/play/1017877/The-Audio-Callback-for-Audio.

...so a "close enough" solution for my game was to:

• Reduce the buffer size (yes, this doesn't exactly solve the problem but it was good enough for my use case).

• Modify the engine source for improved audio latency; here's what I did:

  • For Godot 4.0 see my PRs 38280 and 38210. These PRs are still pending review so use them at your own risk.
  • For Godot 3.2 you can grab a copy of the build I'm using here; it's just my 4.0 PRs backported to 3.2. It should be relatively trivial to rebase my commits to a newer 3.2 version (I may do this eventually at that repo as my game gets closer to launch). This isn't as well tested as the 4.0 PRs because I'm assuming these PRs won't be backported to 3.2, but it appears to compile and work on Windows 10, Windows 7 and Linux.

This was enough to solve the problem in my use case - I hope it's enough to solve yours.

The solution for this would be a new API like Unity's AudioSource.PlaySheduled. @rezgi you may get away with godotengine/godot#32382 (comment) from the related issue @Calinou mentioned.

I like this idea.

[rezgi]

I'm also working on a music/rhythm game with events synced to MIDI data and have experienced the same problem.

There is OS.get_ticks_msec(), if that works for you.

This is exactly what I use, but alone it wasn't enough, because...

The problem is though, that as soon as @rezgi decides to play a sound at an exact moment in time (i.e. at a specific beat), it may be played out of sync by up to n milliseconds (where n is the audio buffer size??) and it may sound awful. Here is a GDC talk about this exact problem https://gdcvault.com/play/1017877/The-Audio-Callback-for-Audio.

...so a "close enough" solution for my game was to:

• Reduce the buffer size (yes, this doesn't exactly solve the problem but it was good enough for my use case).

• Modify the engine source for improved audio latency; here's what I did:

  • For Godot 4.0 see my PRs 38280 and 38210. These PRs are still pending review so use them at your own risk.
  • For Godot 3.2 you can grab a copy of the build I'm using here; it's just my 4.0 PRs backported to 3.2. It should be relatively trivial to rebase my commits to a newer 3.2 version (I may do this eventually at that repo as my game gets closer to launch). This isn't as well tested as the 4.0 PRs because I'm assuming these PRs won't be backported to 3.2, but it appears to compile and work on Windows 10, Windows 7 and Linux.

This was enough to solve the problem in my use case - I hope it's enough to solve yours.

The solution for this would be a new API like Unity's AudioSource.PlaySheduled. @rezgi you may get away with godotengine/godot#32382 (comment) from the related issue @Calinou mentioned.

I like this idea.

Thank you very much for the detailed answer. I also reduced the audio buffer and even set the FPS to 120. It mainly solves the issue but I also wanted to suggest adding a playScheduled feature to Godot for improvement.

[benjarmstrong]

Thank you very much for the detailed answer. I also reduced the audio buffer and even set the FPS to 120. It mainly solves the issue but I also wanted to suggest adding a playScheduled feature to Godot for improvement.

I second the need for a PlayScheduled feature. Even after all my changes I'd still prefer better audio synchronisation in my game.

Another point is my solution doesn't completely solve the problem - it merely reduces it's effects. In projects with complex mixer arrangements it may not be feasible to lower the audio latency below a certain point, in which case my solution isn't very helpful.