Mpv: Denoise Audio with RNNoise

[Nudin]

With RNNoise there is a relatively new, efficient, free/opensource noise suppression technique that does not require training and removes noise from audio very well.
It would be awesome to have RNNoise integrated in mpv, so that by usage of arguments or custom-mapped keys one could reduce the noise of the audio-file or the audio of the video. I would for example love to have this for watching youtube-videos or online-lectures with noisy sound.

RNNoise is available as a C-libary and a lv2-plugin.

[ghost]

There's a wip on having a native lavfi implementation.

[Nudin]

Sorry for the query, but I'm not sure: how will this solve this? Is there a libavfilter doing audio-denoising? Or can libavfilter use lv2-plugins?

[mia-0]

1) libavfilter will get support for RNNoise, meaning that you will be able to use --af=rnnoise with mpv
2) libavfilter can in fact use lv2 plugins if FFmpeg has been built with support for it. See https://ffmpeg.org/ffmpeg-filters.html#lv2

[tobia]

You can use the LADSPA version at https://github.com/werman/noise-suppression-for-voice to use RNNoise with currently distributed versions of mpv and ffmpeg.

That repository is not well organized, so here are some tips:

1. make a directory and run git clone inside it, in order to end up with 2 directory levels (eg. ~/src/rnnoise/noise-suppression-for-voice)
2. edit CMakeLists.txt removing the VST and LV2 entries (we only want LADSPA)
3. edit src/ladspa_plugin/CMakeLists.txt removing build(32) if you are on 64 bit
4. edit src/rnnoise/CMakeLists.txt removing src/rnnoise_demo.c (watch out for the parenthesis)
5. create a link to fix an include path: ln -sf ../include/rnnoise src/rnnoise/src/rnnoise
6. run cmake . followed by make -j
7. check that you got ../bin/x64/ladspa/librnnoise_ladspa_x64.so, which is placed outside the source dir, hence the double level created at step 1.
8. copy it to an appropriate directory and name, such as ~/.local/lib/ladspa/rnnoise.so
9. set up an environment variable pointing to the directory you placed it in: export LADSPA_PATH=~/.local/lib/ladspa
10. launch mpv with --af=ladspa=rnnoise:noise_suppressor, if you installed the library as rnnoise.so as in my example
11. make a Bash alias for quick usage: alias mpv_voice='mpv --af=ladspa=rnnoise:noise_suppressor'

I'm using it to watch noisy lectures, by just typing mpv_voice https://www.youtube.com/watch?v=... and it works very well! Real-time noise removal without the bubbling effect!

[richardpl]

This thing have no parameters to change.

[tobia]

@richardpl It does not.

RNNoise is a research project by Mozilla's Xiph to apply deep learning techniques to noise suppression in spoken audio. The entire premise is to use machine learning to auto-tune the parameters in real time, since the noise profile changes from moment to moment and from recording to recording, in order to get the best voice quality and as few artifacts as possible. As a result, the typical "bubbling" of classic noise removal is gone, as well as the manual training phase.

The result is IMHO much better than traditional techniques, not to mention easier to use (no parameters and no manual training.)

The website has all the explanation and demo files: https://people.xiph.org/~jm/demo/rnnoise/

Here is one of the lectures I'm watching, try it with and without RNNoise: https://www.youtube.com/watch?v=FyoQjkwsy7o

Also, this is not really useful when using headphones, but becomes important when you are using the small laptop speakers, because in some cases the background noise may drown the spoken content and make it hard to hear what is being said.

[richardpl]

I'm working on another denoiser but for libavfilter.

[Nudin]

@tobia Thanks for the cool instructions! I've made a repo based on the on of @werman building in these changes and including a How-To for mpv, including how to map toggling of the filter to a mpv-key (imho even more confinement that an alias):
https://github.com/Nudin/noise-suppression-for-voice

[werman]

Cool to see that RNNoise gains some usage but it's strange that it happens so slow. I'm using it for near a year at it works so well in comparison to denoising in different voice chats.
Didn't know that it is so easy to use with mpv, will take a note!

[richardpl]

Try also afftdn