Godot: SamplePlayer latency - it's just bad performance.

[freemanfromgodotengine]

Operating system or device - Godot version:
Godot 2.1-stable on Xubuntu 14.10 64-bit. Project also tested on Android 4 to 6.

Issue description (what happened, and what was expected):
Delay between action on the screen (for example clicking the button) and hearing the sounds, that is played by pressing the button with something as simple as

func _on_TouchScreenButton_pressed():
    get_node("SamplePlayer").play("sample")

is terribly long.
From my experiments it's around 285 ms.

Latency like that basically prevents people (who don't know how to use low level audio AudioServer), from creating interactive audio games.

[timoschwarzer]

The sound engine will be rewritten for 3.0 I think

[freemanfromgodotengine]

@timoschwarzer I afraid it will not. That was a plan some time ago, but with recent decisions to skip 2.2 and go for 3.0, @reduz aims for the renderer rewrite for 3.0 and not audio rewrite now, but it would be good to hear it from him personally.
I expect it will be done later, but I did the tests and decided to describe audio problems anyway, so we can have reference how better Godot will be in the future.

[ghost]

285ms? Are you sure it's that long? I don't really notice this I think... But obviously it shouldn't be like that, jesus :P

[ghost]

Also, I could be talking about of my ass but is this related to the audio's actual api, or could it be just a signal lag from ui events -> play audio? (input lag. like minuscule)

[beamer159]

@freemanfromgodotengine you mentioned that using the low level audio AudioServer can improve this. How exactly can AudioServer be used, and how much of an improvement does it make?

[freemanfromgodotengine]

I did some additional tests. This time measuring AudioServer's performance. Frankly, I expected much better results.
The best result of AudioServer's latency I was able to get was 235 ms.

I know it's 50ms less then results of SamplePlayer test I did, but from my point of view that is still bad performance.

[ghost]

Wow, this is big. I always thought it was my sound files that were distorted. I usually open them up in Audacity and try to shave off the first part of the sound so it would play faster.... I guess this explains the lag. F*ck

[freemanfromgodotengine]

@beamer159 By using AudioServer I meant using this: http://docs.godotengine.org/en/stable/classes/class_audioserver.html can create voices and samples, and play them. It's also able to change in time parameters of played voices, for example volume, in a much smoother way.
Like I explained in post above, it tends to have less latency than SamplePlayer.

[freemanfromgodotengine]

@Dillybob92 I checked my editing. OK, there was 0.007 ms silence at the beginning of the file in the 1st test I did - the one with SamplePlayer, so it would give us 278 ms instead 285 ms. :)
2nd test - AudioServer test - audio had no silence at the beginning, so 235 ms stands.

[vnen]

How are you measuring this? Have you tried to bypass GDScript (by connecting the signal directly to the play function of the SamplePlayer)?

[freemanfromgodotengine]

@vnen I do it in the most simple and reliable way - "analog". My test looks like this:
I run audio playing Godot project on my computer.
I set external high quality audio recording device, that, using microphones, records everything that comes out of computer speakers.
I bring computer mouse close the microphone and speaker. Mouse has audible click, that microphone is able to record.
I start to record the sounds and I click play" button in Godot project. Godot plays the sound, but my mouse click is also recorded by the microphone.
I copy audio from audio recording device to the computer.
I open audio file in audio editor and look at the time pass between recorded mouse click and sound played by Godot in the timeline (most audio editors provide it and are able to show the time in ms or samples).
That's is.

What signal and how would I directly connect "play" function to it?

[freemanfromgodotengine]

@vnen I did another simple test, just in Godot editor. I went to SamplePlayer inspector and recorded clicking in UI on "play" choosing the sample. Time measured between releasing the mouse button and sound being played was more or less similar to value of 285 ms. So this delay is no created by code. Anyway my test project has nothing more then just TouchScreenButtons mechanism and playing sounds.

[vnen]

@freemanfromgodotengine on your OP you mentioned func _on_TouchScreenButton_pressed() which I assume is a signal connected to that function. With the dialog to connect signal you can select any node and type any function name, so you can select play function from the SamplePlayer node, without any script attached to it.

But since the sample player in editor does the same, I guess there's overhead from GDScript there.

[ghost]

So if I understand correctly, ETA for sound improvements is >6 month? (Considering that 3.0 will come out at late winter/early spring and nobody will care about sound for some time).
I wanted to play around with timed inputs in my game, I guess that mechanic will have to wait until this is resolved.

[freemanfromgodotengine]

OK, I guess it's a test day today :) so I decided to do one more test to check if problem might be related to operating with UI - things like clicking the button with a mouse, or touching the screen of mobile device.

In the test the sound was played on colliding two objects. There was no input from me, just simple code

func _on_RigidBody2D_body_enter( body ):
    get_node("SamplePlayer").play("sample") 

and godot's physics. It was a Rigidbody2d (character mode) bouncing on Staticbody.
I run the test and recorded computer's screen with my camera 30fps.
Then I checked the movie frame by frame.

There was a difference of approx. 6.5 frames between moment of visual contact of the objects and the sound played.
Assuming that 1 frame of video recorded at 30 fps is taken every 0.0333333 of a sec., 6.5 frame delay would give approx. 217 ms . It's better than 285 ms when UI interactions were taking place, but still confirms, more or less, rather noticeable lag.

[SuperUserNameMan]

As I never noticed this delay as a problem, i've made some tests too under Windows 8.1, and compared Godot delay to the delay I'm used to experience in the Left4Dead2 video game.

So I recorded the sound (using a camcorder) near my laptop, then I measured the delay between the sound of the mouse click and the sound emitted by the computer using Audacity.

Under windows 8.1, with a Godot 2.0.4.1 project and default audio settings, I get a delay of approximately 200ms.

With Left4Dead2 into the UI, the delay between the mouse click and the sound of the UI is of around 100ms.
In game, the delay between the mouse click and the sound of the gun is of around 250ms.

[SuperUserNameMan]

around 250ms is what i measured for Left4Dead2 which uses Valve's Source Engine, and for which around 250ms is not ruining the game immersion, IMO.

[volzhs]

The speed of sound in real life, it's 340 m/s.
so, we can hear 250ms delayed sound if something happens about 85 meters out.
Can we get this reality with new audio engine? :)

[bojidar-bg]

Here are some possible workarounds, for those feeling adventurous and not willing to wait till this is fixed:

1. Have absolutely _no_ delay on the .wav sample, since many samples have. (probably you already tried it tho)
2. Have multiple SamplePlayers, and play them every 125ms apart. If no sound is actually needed, stop them, and hope it doesn't degrade the game. (assuming that stopping a sounds takes <70ms, and would go undetected)
3. Test many, many scenarios, until you find just one in which some information reaches the AudioServer and is applied in less than 250ms. For example, if it turns out that changing the volume of something is applied in 50ms, then, run multiple repeating samples of the impact, and tune up one of them when the impact happens, then tune it back down before it starts repeating (-> 50ms sound lag). Or, if pausing then continuing the play of some sample (if it's possible though) takes 100ms to restart the sample, it might be usable for this. And so on...
4. Fork/clone Godot, then start messing around the AudioServer, until you find the real cause of this, then fix it and (maybe) make the whole community happier. In the process, you are also going to gain some nice C++ and bugfixing experience. :smile:
5. Don't add things that require that fast sound response, or fake it somehow. Kind of like early game consoles -- if you can't make real 3D, either fake the 3D or go 2D.

[SuperUserNameMan]

I don't know for the rest of all of you, but when i play the plateformer demo or the simple shooter demo, my gaming experience is okay despite the "200ms" delay ...

[SuperUserNameMan]

@GodotIsAMess : could you upload a minimal project that shows when this delay is an issue ?

[beamer159]

Here is an example project similar to DDR. Use WASD for up, down, left, right. When the player hits an arrow, it plays a sound effect. Ideally, the sound would play immediately when the arrow is hit, but the delay makes this type of game unplayable.

In Tap.gd, look at lines 87 and 119. These lines play the sound effect. Right now 87 is commented out. If you uncomment this line, the sound will play when it is supposed to (without any input from the player)

Tap.zip

[beamer159]

For comparison, look at Osu's Taiko game mode:

https://www.youtube.com/watch?v=kthG-0Un6fA

From experience, I can say that there is essentially no noticeable delay between hitting the key and hearing the sound effect.

[SuperUserNameMan]

@beamer159 : I made a similar demo/game with a early version of Godot 1, 2 years ago, and this delay did not appeared as a problem because, first my game was slower than yours, and secondly because I adapted my logic to workaround the delay.
Even at the speed and precision level required by your game, if you consider the delay is a constant, it should be all about math, IMO.

[beamer159]

I did consider the delay as a constant. That is how I got the sound to play when I wanted at line 87, by playing the sound 200 ms before I wanted it to sound. However, I can't know when the player is going to press the button, so I can't start playing the sound before they press the button.

The problem still stands. I have tried a number of Godot-created games, and in every one, if a sound plays as a result of a player input, this ~200 ms delay exists.

[SuperUserNameMan]

indeed it's not about guessing when the player is going to press the button.
It would be more about making the player synchronizes his reflexes on what happens on the screen ...
There would still be a delay between the user inputs and the sound, but the music and the sounds could still be synchronized.

And if the player wants to synchronizes his reflexes on the music, with some training, his/her brain should just perfectly anticipate the delay ... after all, when a drummer/percussionist wants to synchronize with the rest of the music (like in a cover for instance), he/she has to move his hands and arms before the sound is actually emitted (not just moving his fingers), and his/her brain perfectly anticipates this larger delay ...

I'm not saying that the 200 to 250ms delay should/could not be reduced, nor that it does not pose some little issue/difficulties, but I'm just still not convinced that it is a real handicap that deserve the gigantic warning blinking neon sign that our dear @GodotIsAMess would like to put on the homepage, nor that there is no workaround or tiny compromises possible to keep working on a "6 month long project" ... IMO.

[volzhs]

@GodotIsAMess How about having a blind test with your friends to see if they notice the delay if you have to release soon.

[punto-]

If you think you can fix this then let's have this conversation in your
pull request thread, not here.

On 28 November 2016 at 21:50, GodotIsAMess [e-mail] wrote:

@SuperUserNameMan https://github.com/SuperUserNameMan Well, I was just
saying that because I don't think it's fair for people who have spent 6 +
months (like myself) only to finally realize about this audio bug and now
it's preventing me from publishing the game. Which is very big. It's really
depressing to be honest, but it's the risk you take for using open source
projects I guess. I just don't understand how the main developers here
don't view it as a huge problem but will put off fixing it for Q1-Q2 of
next year. That makes absolutely no sense to me, but I guess I am the
special snowflake here and it only affects my project only. (/s)

Obviously a neon sign on the front page would be dumb, but just saying
that's how I feel personally. Even I were to fix the bug, akien wouldn't
even merge it because he's so delusionally picky about whitespaces. I would
spend more time fixing the damn whitespace in code than actually fixing the
bug in C/C++ code.

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/godotengine/godot/issues/6965#issuecomment-263442758,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AGVmPV5UnxXiAf7-Ln3gEhRyMJ10oE4-ks5rC3a6gaJpZM4KkJRU
.

[freemanfromgodotengine]

Wow, I did not create this issue to make havoc nor bash devs :)
I would like to thank you all for confirming my finds.
I also assume reading the comments, that the issue is relevant for other Godot users.

@SuperUserNameMan When such latency may not interfere with your gameplay in some different genre of games, it will be noticeable or even ruining the experience in others, therefore if we want Godot to be more versatile game engine, the issue should be resolved as much as possible.

I am sorry, but I have to say, that you had no idea what you were talking about when you described brain anticipating the larger delay. It's exactly opposite.
Brains know very well where things should be in time and space and latency even as small as 25 ms is noticed in the sound anticipating cognitive processes. If two sounds will be played with such delay apart, your brain will clearly be able to recognize them as two separate audio events. Latency below that (12 ms) gives just a feeling, that can't be described as separate audio events, but brains "feel" that something is not right.
So, brain compensating for bigger delays can only be described as less and less comfortable task, that probably ends with completely ruining the game.
How ofter did you hear online FPS players with ping 250 ms that were angry and swearing after being killed or not able to headshot enemies because of 250 ms game delay?

Maybe itself it's not yet an issue worth of a gigantic red neon sign saying that "Godot sucks" (because it clearly doesn't), but Godot's audio inefficiencies are evident, as I was trying to state by posting this issue as well as #6963 and #6964

@GodotIsAMess I am glad you see the latency problem as well. It does not make me standing alone here.
However I would constrain myself from blaming Godot devs for your fault of not taking care of choosing the right tool for creating your game. If someone spends building something 6 months with the tool he did not check, it's his own fault if the tool is not the right for the job.
If you say that it's not fair for people who have spent 6 + months only to finally realize about this audio bug, I would say, that Godot's dev's will be glad to fully refund your Godot purchase. ;)

@bojidar-bg about your list of possible workarounds.

• ad 1. wav files lag as well.
• ad 2. does actually nothing. May work in space shooters, but not in myriads of other games and sound solutions needed in games. Also who knows what would be the performance cost.
• ad 3. Before posting this issue I tested as much as I could, learning Godot and looking for solutions... since Godot version 1.0.
• ad 4. I wish I would be able to contribute on such lvl. I even started to learn C++ at some point, but I am afraid I am too slow or C++ is too hard... at least for now. But you might be true. I might need to stop reading books and do exercises and jump into code from github, trying to understand and fix something real.
• ad 5. Not everything can be faked. The lack of proper timely response between player's action and game's reaction in sound or visual feedback is really easy to spot by an average human being. And some games just need or even are created around obligatory fast sound response. BTW early game consoles and 8-bit computers had better latency than 250 ms ;)

@volzhs they will notice the delay, unless the game will have a strange, visually misleading mechanics or something else, that would be able visually justify the reaction latency. It's possible, but needs creativity in game design :)

@punto- @bojidar-bg I am sorry to say this, but no Godot developer should expect Godot users to learn C++ and help with the engine problems etc.. After all GDScript was created to be simpler and easier to learn. It's logical, that Godot will attract many users who were not able to learn other, more complicated and difficult languages, frameworks, game engines. Telling them that they should have the conversation about problems in their pull request thread is not the way to go imho.

[SuperUserNameMan]

@punto- @bojidar-bg I am sorry to say this, but no Godot developer should expect Godot users to learn C++ and help with the engine problems etc.. [...] Telling them that they should have the conversation about problems in their pull request thread is not the way to go imho.

( for the sake of minding others' business and of subtly making the unobvious more obvious, i just would like to point out that the GodotIsAMess' message to which Punto replied by mail is not exactly the same as the one currently displayed in github .... In the initial version, there was a second paragraph :

Even I were to fix the bug, akien wouldn't even merge it because he's so delusionally picky about whitespaces. I would spend more time fixing the damn whitespace in code than actually fixing the bug in C/C++ code.

.... )

[bojidar-bg]

@freemanfromgodotengine Well, our response about "go fix it yourself" was result from @GodotIsAMess's ramble on another (totally unrelated) issue. As we told him multiple times, this is planned to be fixed when the audio engine is rewritten (or just refactored) in 3.0/3.1.

Till then, we can't really suggest something else to people whose projects are blocked by this, as devs are giving away their free time on the issues they freely choose from -- and if they think something is more important than this, we can't really blame anyone.
Anyway, if you feel adventurous (not forcing anyone here), just go in there, find every C++ function which is called, print_line the millisecond time in all of them, and identify the delay somewhere in the pipeline -- this would be really helpful for a swift fix. :smiley:

[punto-]

@freemanfromgodotengine I don't expect anything from anyone, but my advice
is that if you want to be a successful game developer of course you need to
learn c++, especially when nobody can fix your issue after complaining so
loudly about it. Even AAA studios with millions of dollars in budgets
prefer to have in house people that can fix their engine issues, instead of
depending on a 3rd party to solve their problems. We do our best, and as I
said on the other thread I agree that this is a problem, and ideally I'd
like to see it fixed, but we can't be responsible for your game. Especially
if he brags about being able to fix it, why are we having this discussion?

On 5 December 2016 at 12:13, Rémi Verschelde [e-mail]
wrote:

Remove above comments that went a bit far from the initial topic (+ edited
@SuperUserNameMan https://github.com/SuperUserNameMan's answer to add
the proper quote that makes it more understandable).

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/godotengine/godot/issues/6965#issuecomment-264879367,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AGVmPSZLGd2X2V1AWKq2QESnVVpCpatCks5rFCoSgaJpZM4KkJRU
.

[leezh]

Okay, so I dug around a bit, and the audio driver buffer size is based on audio/output_latency in the project settings. By default it'll be 25ms @ 44100Hz, giving 1102.5 buffer size but because it's rounded to the next power of two, it'll actually be a 2048 size buffer or 46.4ms latency. What about the audio mixer? Same principle with audio/mixer_latency, but this time 10ms is actually 11.5ms. Not too bad.

All in all, buffer size alone, you're talking about 57.9ms latency.

Now comes the interesting part. In main/main.cpp I saw that the main thread operates by processing Input -> Rendering -> Audio Mixer -> Script. That means, from mouse click to actual output, it has to go Input -> [Rendering] -> [Audio] -> Script -> [Input] -> [Rendering] -> Audio. Two whole frames. At say, 60fps, that's 33ms. At the captured video 30fps it's worse, at 66ms.
EDIT: Digging a bit more and maybe I might be wrong here, but it should at least be 16-33ms.

Now, because the audio driver is updated in a separate thread you'll get a vsync equivalent, giving an overall total of 33 + 58 + (+46.4 / -0) = 114.2 ± 23.2 ms.

So yeah, quite a bit of wastage that can be shaved off with some tweaking.

The other 100ms I'd say is down to OS latency in the audio system (~10-50ms unless you have a realtime kernel patch), input (wireless mouse?) and stuff I can't account for without actually touching the code, which I might get around to at some point.

[leezh]

Just tested it myself, using my phone as a mic. For control, my old laptop has a 400ms (!! it's old though) delay. Reducing the audio/output_latency to 20ms reduced it to 350ms. Seems like that variable isn't working 1-to-1 to actual latency. Possibly a race condition somewhere?

EDIT: Arrrgghhh, I realised I forgot to set click-on-press. No, and with repeat testing it seems to only be shaving off 5-10ms

[bojidar-bg]

Nice debugging work, @leezh! I guess you are from the "feeling adventurous" group, no? :smiley:

[leezh]

Haha, yeah. I have very creative yet oddly productive ways of procrastinating actual game making. XD

Anyways, I found that doing one or two tests wasn't enough so I recorded multiple tests :P (n=8):
25ms: x̄=280ms σ=26ms
20ms: x̄=244ms σ=30ms

Seems like my earlier prediction was right! That said, it's only a small improvement in the grand scheme of things.

Now, before you go about reducing the output_latency to something ridiculously small, I should warn that if it's too low you'd get lots of popping sounds during playback if the CPU can't keep up.

[freemanfromgodotengine]

Nice experiments @leezh
I played with audio buffer settings to. Like you, I noticed, that it's able to set the latency back only a little and can cause other sound related problems, especially on mobile.

[punto-]

That's awesome (I guess I was wrong when I said it'd involve seriously
breaking the AudioServer, only the main loop :p ). Does it make any
difference if you play the voice from the _input callback instead of
_process? I don't remember if the dispatch order of those events is assured
to be the same across all platforms..

On 6 December 2016 at 11:21, Freeman [e-mail] wrote:

Nice experiments @leezh https://github.com/leezh
I played with audio buffer settings to. Like you, I noticed, that it's
able to set the latency back only a little and can cause other sound
related problems, especially on mobile.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/godotengine/godot/issues/6965#issuecomment-265161020,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AGVmPTo4CaZwMGXi-CUgo4PWRafwAatDks5rFW9cgaJpZM4KkJRU
.

[freemanfromgodotengine]

@punto-
Some time ago @vnen was asking about

"Have you tried to bypass GDScript (by connecting the signal directly to the play function of the SamplePlayer)?"

so I

"I went to SamplePlayer inspector and recorded clicking in UI on "play" choosing the sample. Time measured between releasing the mouse button and sound being played was more or less similar to value of 285 ms. So this delay is no created by code. "

I guess if such latency exists in the editor itself when I click on the SamplePlayer node to play the sample, it is not related to the _input nor _process, right?

[leezh]

@freemanfromgodotengine The editor is built using the same GUI tools provided, so it definitely would be related to _input()

@punto- So, yeah, after finally digging around more I am starting to better understand how the main loop is set up. In platform/x11/os_x11.cpp the input is polled and _input() dispatched right at the start of the frame. Only then does it go through the regular stuff in main/main.cpp. In other words, it's Input -> _input() -> Rendering -> Audio. That's one frame of delay. Moving AudioServer::update() to before VisualServer::update() should improve things.

Also, to take into account OS latency that can't be dealt with, I did some research and found that they're typically around 20ms total. Heck, I even tried using aplay on the command line with a tiny wav file, and got only a 50ms delay.

In other words, we're actually still looking at a fairly large chunk of latency that we don't know the cause of. :/

[bojidar-bg]

@leezh Again, I'd guess it is some thread-locking and syncing that's bottlenecking here... But that would be so only if AudioServer is multithreaded (might be wrapped in a thread as well).

Also, I'd guess, we should test directly dispatching AudioServer from GDScript in _input. That way, we can be sure if the latency is from the sampleplayer or somethere else :smile:

[SuperUserNameMan]

btw, how long does it takes to Android (and also to other OS maybe) to tell the difference between a simple touch/press/click, a double tap/click, a swipe, a drag and drop, and a middle finger triple backflip click, etc ?

edit : nevermind, Android will send onTouch events when they occurs, and will detect gesture in parallel, so its not blocking.

[lonesurvivor]

So I am very annoyed by this problem and messed around with it today. I measured the times at some points by printing out the result from OS::get_ticks_msec. By doing this, I found out that there are two bottlenecks:

1. The time that passes between the call of AudioServer::voice_play() (which happens almost immediately after receiving the triggering _input event) and the queued CMD_PLAY command getting processed within AudioServerSW::driver_process_chunk() is about 10-50 ms. Which is the less significant amount of the whole 200-300 ms delay.

2. After tracking it down to the lowest level, where the driver is being handled, I came to the comclusion that the problem must be there. And I was right.

I am using Linux with the PulseAudio driver. After verifying that AudioDriverPulseAudio is actually being used, I played around with that file a bit, finding out that there is function which returns the latency from PulseAudio. Printing it out showed a latency as high as like 200 ms, just as big as expected. Reading about the PulseAudio API, I played around with the initialization parameters, especially with the buffering attributes pa_buffer_attr, which currently are set to NULL for default initialization.
I then set the pa_buffer_attr::tlength to a low value, compiled, and... The delay was gone! So the problem is a too large set buffer size. On the project page of PulseAudio (http://0pointer.de/lennart/projects/pulseaudio/doxygen/structpa__buffer__attr.html) it says that

The server tries to assure that at least tlength bytes are always available in the per-stream server-side playback buffer. It is recommended to set this to (uint32_t) -1, which will initialize this to a value that is deemed sensible by the server. However, this value will default to something like 2s, i.e. for applications that have specific latency requirements this value should be set to the maximum latency that the application can deal with. [...]

So it seems that the server sets that to a very high value by default, which causes the high latency. Using the global settings value "audio/output_latency" for the tlength setting (which is being loaded in that file anyway), I managed to fix this problem within a few lines of code. The buffer size does now depend on this setting, which is by default 25 ms (which is totally fine).

So I made a pull request, but also decided to put it up here for discussion. While I was able to fix this, I didn't really know what I was doing (which is probably normal when you are trying to fix bugs in a huge unkown code base). And it would fix the problem only for pulseaudio. I will take a look into ALSA and maybe the windows driver too later, but I definitely can't look into the iOS, Android and Mac drivers (since I can't test it). It is a bit weird anyway that this problem occurs with all of the audio drivers which are independently form each other. So what do you think about this?

Edit: pull request is #7425

[bojidar-bg]

@lonesurvivor Yay! Finally someone adventurous enough to want to do this! :+1:

__Seems__ like ALSA already does most of the buffer thing: https://github.com/godotengine/godot/blob/master/drivers/alsa/audio_driver_alsa.cpp#L99
What is weirder is that RTAudio also seems to be doing it, though it still suffers from the bug :open_mouth:
https://github.com/godotengine/godot/blob/master/drivers/rtaudio/audio_driver_rtaudio.cpp#L138

[lonesurvivor]

The difference with ALSA is that the output_latency setting which is used there has actually an effect on the delay (unlike with PulseAudio). If you set it to like 10 ms or less, the delay is nearly gone. But if you set it to 50, it is already like half a second. With 100 it is even more, nearly a second ( all measured by hearing). So it seems that there is a multiplicative factor somewhere in there. I will take a look at this tomorrow.

[timoschwarzer]

@bojidar-bg I've done a quick search on RTAudio and the documentation at https://www.music.mcgill.ca/~gary/rtaudio/classRtAudio.html#a6907539d2527775df778ebce32ef1e3b says that the buffer is measured in sample frames. I don't know if those are milliseconds but I think they aren't.
Same for ALSA: http://www.alsa-project.org/alsa-doc/alsa-lib/group___p_c_m___h_w___params.html#ga9162045265f283c532634506456cab09
PulseAudio has the tlength field, which is temporal.

[SuperUserNameMan]

@timoschwarzer and @lonesurvivor :
I don't remember all the things i tried, but just to share a little of my last month experiments about RtAudio with you, i tried various settings for RtAudio::StreamOptions options; without noticing any improvements.

I think that options.flags should at least be set to this :
options.flags = RTAUDIO_MINIMIZE_LATENCY | RTAUDIO_SCHEDULE_REALTIME;

Reducing unsigned int buffer_size tends to lead to buffer underflows, and increasing the number of options.numberOfBuffers = 16; did not improved things neither.

I hope you'll find the solution.

I tested on Windows 8.1.

edit : also, I measured the latency between the hardware click-sound of my USB wireless mouse, and the sound emited by my computer. So, my latency equation was something like this : mouse wireless latency + receiver wireless latency +USB latency + Godot latency + Rtaudio lantency + audio driver latency + hardware latency

[lonesurvivor]

@bojidar-bg
ALSA is fixed as well now(at least it works as it should for me now). It turned out that the buffer size wasn't set at all, instead the period size was set by the output_latency setting (in the line you linked above). Added it to the pull request.

[bojidar-bg]

@lonesurvivor Nice progress so far, do you think you can handle RTAudio as well? :smiley_cat:

[lonesurvivor]

I will at least take a look at it. At the moment compiling with support for RTAudio on Linux is failing, need to get that going first.

[lonesurvivor]

I guess I found the problem with RtAudio. It's a misconfiguration again:
The setting of the buffer size works fine here, but RtAudio uses multiple buffers. The buffer size argument of rtAudio::openStream() sets the size of each of those buffers. Every time the callback function is being called by RtAudio, only one of those buffers is filled with samples. After that, the next buffer will be selected to be filled next time. They seem to be arranged in a ring like structure: The sound system plays the samples of the oldest one and moves on to the next one. After that, the buffer is ready to being overwritten with new samples. So the sound output is behind not only the buffer_size but buffer_size * buffers. So if there are, for example, four buffers and the buffer size is set to 512 samples, the latency is about 2048 samples.

This is what you get when you print out the latency calculated by RtAudio (with getStreamLatency()) every time the callback function is called:

...
audio buffer size: 512
actual audio buffer size: 512
actual channels: 2
latency: 0 bufferFrames: 512
latency: 512 bufferFrames: 512
latency: 1022 bufferFrames: 512
latency: 1531 bufferFrames: 512
latency: 2041 bufferFrames: 512
latency: 2069 bufferFrames: 512
latency: 2057 bufferFrames: 512
latency: 2085 bufferFrames: 512
latency: 2076 bufferFrames: 512
latency: 2064 bufferFrames: 512
latency: 2094 bufferFrames: 512
latency: 2084 bufferFrames: 512
latency: 2076 bufferFrames: 512
latency: 2065 bufferFrames: 512
latency: 2092 bufferFrames: 512
...

You can see the latency starting at 0 after initializing the stream and then ramping up to about four times the set buffer size of 512 samples. The buffers are getting filled, and after that the audio stream starts playing and the latency stays more or less constant.
This is what happens in the current state of the RtAudio driver. So it seems that RtAudio uses four buffers per default. It is possible to set the amount over the options argument. However, as stated at the website (https://www.music.mcgill.ca/~gary/rtaudio/settings.html), it isn't guaranteed that you get the amount you wanted (same with the buffer size). So it could vary from system to system, which makes it problematic. A fast fix for this would be to set a fixed amount of buffers and then divide the given output_latency by it. But what if some systems don't support the setting and want more buffers? Then the set latency is wrong again. In my case, the lowest possible and automatically set value is 4, but this corresponds to only 100 ms latency with the default output_latency value of 25 ms. But some posts above mention a delay of more than 200 ms, which would correspond to 8 buffers.

I could just set this to 8 buffers and hope that this is ok with most devices. But it might not be. The website also mentions that the setting is ignored by certain audio systems.

So what do you think?

[bojidar-bg]

@lonesurvivor Wouldn't it be possible to get the number of buffers via their apis? For me it seems like RTAudio::StreamOptions::numberOfBuffers is want you want.

[lonesurvivor]

Yeah, that is kind of how I did it now. What is not so cool about this though, is that the stream has to be opened to get the actual number. And you can't modify the buffer size after it has been opened. So if the desired numberOfFrames and the actual number are not equal, the stream has to be closed and reopened with the appropriate modified buffer size.
But I implemented it and it works fine for me. Pull request will be updatet in a second. If possible, please test this under windows (and Mac?).

[bojidar-bg]

Good job!

To everybody on this issue -- if you want the bugfix to get through faster, it would be pretty cool if you can test the #7425 PR, especially on non-linux platforms (but linux testing is also welcome :smiley: ). @freemanfromgodotengine @GodotIsAMess @SuperUserNameMan @beamer159 (and others I didn't spam... yet)

[lonesurvivor]

RtAudio is quite annoying under linux though (at least for me). Messes up my sound all the time and fails to initialize a lot. There is a reason that it is not included in the build on linux by default... PulseAudio and ALSA work way better.

However, I will take another look into the command queue which I mentioned above. Maybe the time between issuing the play command and it getting processed could be shortened as well.

EDIT: Looks like there isn't much to be improved there. The delay there depends on the hardware latency since the command processing function is called everytime the driver pulls more samples. So moving the start of the playback directly to the voice_play function wouldn't do anything. So it looks like we are done now. I did what I could, so I hope it is now fixed for everyone.

[akien-mga]

As I understand it, #7425 should have fixed this issue for PulseAudio, ALSA and RtAudio. I guess the only part left is to check Xaudio2 and maybe the OSX driver: https://github.com/godotengine/godot/blob/master/platform/osx/audio_driver_osx.cpp

[akien-mga]

Given the length of this issue and its half-fixed state, I would propose that some of us test the current master branch on all platforms they can and try to assess whether the SamplePlayer latency is correct or not, then we could open a new issue for the platforms/drivers which are still not behaving properly yet.

[akien-mga]

I've opened #7603 to keep track of tests of the fixed drivers and the other ones, so let's close this one.