Mumble: Echo cancellation - explain difference

[toby63]

Context:
Explain functions to user.

Description:
I noticed that it is not explained what the difference is between:

• mixed echo cancel
• multichannel echo cancel

So maybe a tooltip with a little explanation could be added, to help users decide which one to choose.

See also: #4127

[streaps]

1.) How does multichannel echo work? Is it for multiple mic inputs?

2.) Does anyone use it? Is it really needed?

3.) Is it specific to speexdsp? And when will speexdsp be replaced by something better (as it's not very good).

[Krzmbrzl]

@davidebeatrici do you know something about this?

[toby63]

In addition:
Is the FAQ still valid?
It mentions the following:

• On Linux, we support echo cancellation when using PulseAudio
• on Windows XP we support echo cancellation using ASIO (which unfortunately requires a very high quality soundcard with ASIO drivers).
• On [Windows] Vista, we support echo cancellation for any sound card.
• There is currently no Echo Cancellation in the OS X client.
  We've tried to use the VoiceProcessingIO built-in echo cancellation (which is what the Mumble for iOS), but it doesn't seem to work to well on OS X, at least it wasn't obvious how to get it to work the last time I tried.

Source: https://wiki.mumble.info/wiki/FAQ/English

Further questions:

• What is with newer windows versions?
  Same as Vista I assume?

[Krzmbrzl]

At least Linux seems to still be valid

Same as Vista I assume?

I guess so too. I can't really imagine anything else

[fedetft]

Didn't know that this question was asked just a few days earlier.
I guess it was the right time to improve echo cancellation.

Anyway, I would like to add something to this topic. As I don't like black boxes, I had a quick look at libspeexdsp's multichannel echo cancellation algorithm, to understand what use it makes of the additional data that gets passed to it.

It turns out it does something simple but reasonable. For each speaker channel you give to it, it tries to independently detect its echo contribution to the input and cancel that.
From this understanding, I think a FAQ can be added to the mumble wiki to more in-depth explain how to choose the echo canceller:

"""
Should I use mixed or multichannel echo cancellation?

Mumble uses acoustic echo cancellation (AEC) from libspeexdsp. Echo cancellation works by finding a copy of the signal that is played through the speakers/headphones in the microphone, and cancelling it. To do so, it must estimate the echo delay, which is the time the played back audio takes to be heard back in the microphone.

Mixed echo cancellation works by mixing the signal from all your speakers in one mono channel, and using that for echo cancellation. As the echo canceller only has to find one signal, this option uses less CPU. This is the best option if the echo delay from all your audio outputs to the microphone is the same, or very close. This is the case when you are using headphones or speakers very close together, such as the built-in speakers of a laptop/TV/sound bar.

Multichannel echo cancellation tries to independently estimate and cancel the echo from every speaker output to your microphone. For this reason, it requires more CPU. Multichannel echo cancellation should work best in a setup where there are significant distance variations between speakers and the microphone, such as when speakers are placed at the corner of a room, or in a full 5.1 speaker setup.

Note: up to mumble 1.3.0

• the echo cancellation was very basic, and it only cancelled periodic signals, but not voice
• the multichannel echo canceller was not operating at all due to a bug. If you are using mumble 1.3.0 or earlier, use mixed echo cancellation or better still, upgrade to a newer version.
  """

The last sentence may need to be removed until a version with the echo cancellation working gets released, but I would definitely want to inform users that a better echo canceller is in the works.

[toby63]

@fedetft
Very interesting and thank you for the explanation :smiley:.

that a better echo canceller is in the works.

I assume it refers to this #4167 ? Ah I see, further work needs to be done, so multichannel is not working for now, but mixed is working better (including voice cancel)?

By the way: could you also help with #4127 ?

@Krzmbrzl
https://github.com/mumble-voip/mumble/pull/4167#issuecomment-629075641

So am I assuming right that "multichannel echo cancel" is not working at the moment?
Maybe a notice about this should be included in 1.3.1 or the multichannel option even deactivated.

[fedetft]

Well, if there's a plan to release 1.3.1, why not include the fix directly rather than deactivate the option? (of course assuming the maintainers like my patch).

[toby63]

@fedetft Is there a fix already?
For my part the release could be delayed until then :wink: (FYI I heard the release is only days away).

Also:

My branch is called webrtc because the original intention was to replace mumble's echo cancellation algorithm, which to me at least performed poorly, with webrtc's Google AEC.
[...] libspeexdsp's echo canceller isn't bad at all, it was just not operating due to a precondition violation.

So you would stay with libspeexdsp?

[fedetft]

Multichannel fix has been committed. Both work and cancel voice echo now.

Yes, I would stay with libspeexdsp. By the way, if you are set up to build a development snapshot of Mumble you can try my branch even before it gets merged and let us know what you think of the new echo canceller.

I did a test with a friend, both using my branch, and we could hold a conversation with no echo and no larsen effect despite both using speakers (not headphones) and no push to talk.

You just need to adjust the audio (speaker) volume so as not be too loud to clip. The echo canceller can't cancel nonlinearly distorted echoes such as clipping.

[Krzmbrzl]

We plan to backport the changes of @fedetft into the 1.3.x series before releasing 1.3.1 :point_up: