Librosa: Allow for streaming per-channel energy normalization (PCEN)

[daemon]

Description

AFAIK, the current implementation of PCEN (librosa.core.pcen) is non-streaming. This is an issue, since applications in, for example, speech recognition and keyword spotting depend on streaming computation of spectrogram features. My PyTorch PCEN implementation supports streaming audio (and trainable weights), but it's probably slower on the CPU than LibROSA's implementation.

[bmcfee]

Thanks for posting this.

You're correct that pcen, as well as just about everything else in librosa, does not directly support streaming. Because pcen specifically relies on IIR filtering, it's not as straightforward to implement as, say, melspectrogram in a block-streaming fashion. However, I wonder if exposing the initial filter state (zi) as a pass-through parameter to lfilter would be enough to let you stream data through pcen in a coherent fashion?

@lostanlen got any thoughts on this?

[lostanlen]

Thanks @daemon for raising this issue and for sharing a PyTorch implementation of PCEN under MIT license! :)

I don't have any experience with streaming, but yes this would be a great addition. I guess that the way we close this issue depends on the decisions we end up taking re: #101.
I agree with @bmcfee that hooking into the initial filter state (zi) is the way to go for the IIR filter. Very neat idea 👍