Mumble: Noise gate generating packet loss

[blablablerg]

Hi, I and my friends have discovered that applying a noise gate before the audio input to mumble seems to generate significant packet loss and possibly introduces short static noise bursts into the stream ('kchchk kchchk' sounds). I haven't fully verified the static noise part, so this ticket is about the packet loss, but we suspect that noise bursts are introduced because of the packet loss. This is especially noticeable at higher bandwidth settings in Murmur (bandwidth = 240000 in Murmur.ini). This is the only setting I have changed in murmur, the rest is default.

I have attached two samples, one with a noisegate applied and one without:
noisegate_bug.zip

This is when playing back the sample through a virtual audio cable with no noise gate, it is just the room recorded with my microphone:

This is when the noisegate is applied. I used Reagate (noise gate from Reaper) on the lowest attack and release setting, with the threshold at the noise floor to make the noise gate open and close at a high frequency. The less frequent the noise gate opens and closes, the less packet loss I see.

Steps to Reproduce
Steps to reproduce the behavior:

1. Set bandwidth to 240000 in murmur.ini
2. Start murmur on localhost to eliminate the network as a possible confounder
3. Route the audio output of your PC to mumble with a virtual audio cable, and play back both attached samples.
4. Check UDP network statistics in the connection information window. You should see a difference in packet loss From Client.

Alternatively, you can check live input with and without a noise gate applied to the stream, with low attack and release settings to make the noise gate 'stutter'.

My mumble audio input configuration:

Expected behavior
No packet loss

Desktop (please complete the following information):

• OS: e.g. Windows 10
• Version: Mumble 1.3.2 and Murmur 1.3.2

Additional context
A friend of mine can also reproduce it on Catalina.

[Krzmbrzl]

Does the problem also occur with different bandwidth settings?

Is the used server the same version as the used client?

Is a noise-gate in principle just a noise suppression system? Aka it let's audio through that it thinks is not noise but blocks audio that it considers noise?

Have you tried increasing the audio per packet and/or decreasing the quality?

[blablablerg]

Hello Krzmbrzl,

Thanks for your quick reply.

Does the problem also occur with different bandwidth settings?

At lower bandwidth settings the problem is reduced (as stated in my original report).

Is the used server the same version as the used client?

Yes, as stated in my original report.

Is a noise-gate in principle just a noise suppression system? Aka it let's audio through that it thinks is not noise but blocks audio that it considers noise?

A noise gate attenuates the audio signal below a certain dB threshold, see https://en.wikipedia.org/wiki/Noise_gate. That is the audio effect I used. A noise suppression system is in my experience a more marketing kind of term, it can mean multiple things.

Have you tried increasing the audio per packet and/or decreasing the quality?

No, I haven't, because I am not looking for a "quick and dirty" fix. That would be for me to simply not use a noise gate and accept low volume noise passing through. However, I find it very strange that audio information in itself can produce significant packet loss on a connection no matter the settings, and I thought it worthwhile to report!

So I am interested if you guys can reproduce it and if yes, hopefully fix it, because a noise gate is a very handy and commonly used effect in (live) vocal recording and processing. This behaviour might also impact audio quality in other unforeseen ways.

[mcdowell84]

Hi, I have reproduced the packet loss bug using only the mumble internal noise gate at maximum settings, no external noise gate.
All other settings are identical.

Mumble Noise suppression set to maximum results in 8.5% loss, during this short test:

Mumble noise suppresion turned off, zero packet loss:

[streaps]

I wonder if this is real packet loss or if packets just don't get send by the client intentionally when there is silence (even in continuous mode).

[Krzmbrzl]

No, I haven't, because I am not looking for a "quick and dirty" fix.

@blablablerg This is not about finding a "quick and dirty fix". It's about figuring out what is going on so that eventually we might be able to fix it. So please go ahead and test it :)

Hi, I have reproduced the packet loss bug using only the mumble internal noise gate at maximum settings, no external noise gate.
All other settings are identical.

@mcdowell84 By "noise gate" I assume you are referring to the "noise suppression" feature of Mumble in the Audio Input settings? Which Mumble version are you using?

I wonder if this is real packet loss or if packets just don't get send by the client intentionally when there is silence (even in continuous mode).

I was kinda thinking the same but this would be weird as the fact that it is shown as packet loss indicates that the packet numbering still increases so that the receiving client receives jumps in the packet's number. It could be however that the increasing of the counter is done before it is decided that the packet isn't actually going to be sent... :thinking:
This however wouldn't explain why the server's bandwidth setting seems to have influence on the issue :thinking:

[mcdowell84]

yes, noise suppression feature, under audio processing, set to -60dB and OFF.

version 1.3.2

[Krzmbrzl]

Alright thanks - will try once I find the time :+1:

[Rickvlaar]

I was kinda thinking the same but this would be weird as the fact that it is shown as packet loss indicates that the packet numbering still increases so that the receiving client receives jumps in the packet's number. It could be however that the increasing of the counter is done before it is decided that the packet isn't actually going to be sent... 🤔
This however wouldn't explain why the server's bandwidth setting seems to have influence on the issue 🤔

Also there is degradation of quality, so it seems that some packets aren't sent that do contain some audio.

Interesting to note is that issue occurs both with the internal noise suppression feature and third party noise gate software. Also we noticed decreasing the 'audio per packet' setting increases the number of packets lost.

I'm the friend mentioned in the bug btw ;) Thanks for checking this out!

[mcdowell84]

Latest opus build contains a bugfix for something which seems related, discontinous transmission (DTX) of digital silence.

https://opus-codec.org/release/stable/2019/04/12/libopus-1_3_1.html

"This Opus 1.3.1 minor release fixes an issue with the analysis on files with digital silence (all zeros), especially on x87 builds (mostly affects 32-bit builds). It also includes two new features:

A new OPUS_GET_IN_DTX query to know if the encoder is in DTX mode (last frame was either a comfort noise frame or not encoded at all)"

[Krzmbrzl]

Hm... In theory though the codec itself shouldn't affect packet loss as this happens in the networking layer. Unless Mumble counts invalidly encoded packets as "lost" :thinking:

[mcdowell84]

Yeah it's very puzzling. Something seems to be coordinated, and opus has packet loss features and packet loss concealment.

Opus developer says "This is only useful is you're writing a VoIP/videoconferencing application and you've asked Opus to stop transmitting when there's no speech. "

https://hydrogenaud.io/index.php?topic=117526.0

[fatexs]

I can confirm this, I am running Mumble on a local network <1ms ping. I did a UDP transmit test with iperf and 500Mbit bandwidth and had 0,0% packet loss.

However if I have Noise Reduction enable in Mumble Client, my packetloss gets reported at 24% but I get perfectly understood and my quality is good.

That suggests only non vital packages get discarded. Disabling Noise reduction reduces my packetloss to 0%
Client 1.3.2 on Windows 10
Server 1.3.2 on Arch linux

Edit: this is with server bandwidth at 128kbit/s so not much higher than default.

[Rickvlaar]

The quality degradation is not so bad that you won't be understandable anymore, but it does degrade the quality in a way that is disruptive.

This is a recording of the static that is generated by the issue:
static_noise_example.wav.zip

Recording made in Mumble 1.3.2 as uncompressed .wav on 2020-08-26

[mcdowell84]

I can also cause the same issue to appear if I turn the gain on my XLR interface (in which the mic is plugged) all the way down to 0 (thus causing silence):
It causes massive packet loss, and other people on the server can hear the scratchy audio artefacts ('kcchhk kcchhhk' sounds).
This is without any noise gate settings within mumble or on the computer.

So it seems that using an external volume control can also cause packet loss, which is very counter-intuitive :)

[Krzmbrzl]

I'm pretty sure that the issue is indeed not the noise gate per se but rather the way Mumble handles silent audio frames. There must be some logic (probably something that tries to be smart and to save bandwidth or whatever) that kicks in in these cases but doesn't recover properly from it :thinking:

[mcdowell84]

yeah, or in how it interfaces with opus
the opus codec also has bandwidth saving features

[RX14]

I'm on mumble 1.3.2 and several people using rnnoise in the server i frequent have noticed this bug: hearing a very short period of static very infrequently. Having it be linked to packets with absolute silence makes sense.

[TerryGeng]

I remember the mumble uses a sequence number mechanism to detect if there's any missed packet. A possible cause for the "packet loss" may most possibly not a network fault, but the internal logic breaks somewhere.

I just want to make sure it is not caused by some delay caused by the audio process chain. Did you notice any kind of audio glitch or high CPU usage on the client PC where the noise gate is applied?

[reelsense]

I think I have the same issue. Try mumble 1.3.0 or 1.2.19, Mumble 1.3.1 and greater have the issue.

When I do a continuous broadcast on on _any_ audio interface except my built in mic I get egress only packet loss.

[blablablerg]

@TerryGeng No, no high CPU usage or anything. You can try it out yourself if you want, I have included some audio files and a description on how to reproduce.

[reelsense]

I solved my client side egress only packet loss by using 1.3.0. Out of curiosity is there any transcoding done on the server?

[Krzmbrzl]

Out of curiosity is there any transcoding done on the server?

No. The server only relays the audio packets.

I solved my client side egress only packet loss by using 1.3.0.

Okay that's interesting. Does this mean that the first version that introduced the issue for you was 1.3.1? Or did you not test that one?

[reelsense]

Okay that's interesting. Does this mean that the first version that introduced the issue for you was 1.3.1? Or did you not test that one?

I tested 1.2.19, 1.3.0, 1.3.1 and 1.3.3 clients.

If I use ≥1.3.1 and I enable _continuous_ transmission my outbound packet loss will immediately be 4-25% and from the receivers perspective it looks like I might have voice activation enabled, but on my end it's clearly a continuous transmission.

When I record mumble audio with software like Audio Hijack it scrapes the application audio and records the missing segments of my transmission as the receiver is evidently hearing it... I found 1.3.0 to solved this.

---

Recent Anecdote:
I've hosted mumble servers for 10 years, I've experience partial and full network outages, clients with bad wireless coverage, etc. The following experience seemed similar to the issue I just described above:

I recently recorded mumble audio when both clients were running 1.3.0 and the server is running the 1.3.3 PPA for Ubuntu 18.04 or 20.04. The reason I ask if there is any transcoding done on the server is, I actually experienced a similar episodes of missing audio transmission but only from the server to the client this time. But my recording method captured my mumble client audio with my full transmission. I connected with the mumble mobile client as an additional test and it looked like the continuous transmission was voice activation. Could this be an issue with 1.3.3 on the server too?