Web-audio-api: (AudioBufferSourceNodeResampling): AudioBufferSourceNode resampling

[olivierthereaux]

Originally reported on W3C Bugzilla ISSUE-17377 Tue, 05 Jun 2012 11:59:42 GMT
Reported by Philip Jägenstedt
Assigned to

Audio-ISSUE-92 (AudioBufferSourceNodeResampling): AudioBufferSourceNode resampling [Web Audio API]

http://www.w3.org/2011/audio/track/issues/92

Raised by: Philip Jägenstedt
On product: Web Audio API

AudioBufferSourceNode has a playbackRate attribute which will require interpolation/resampling of some kind. However, it is not defined how to resample. Possibly it should be as close as possible to an ideal resampling, in which case that should be stated. Alternatively, it could be possible to specify which kind of resampling to perform via an attribute: nearest, linear, cubic, sinc, etc...

It also needs to be defined what should be done about folding when the net result of the sample rates and playback rate is a downsampling, if anything.

[olivierthereaux]

Original comment by Chris Rogers on W3C Bugzilla. Tue, 16 Oct 2012 20:27:50 GMT

(In reply to comment #0)

Audio-ISSUE-92 (AudioBufferSourceNodeResampling): AudioBufferSourceNode
resampling [Web Audio API]

http://www.w3.org/2011/audio/track/issues/92

Raised by: Philip Jägenstedt
On product: Web Audio API

AudioBufferSourceNode has a playbackRate attribute which will require
interpolation/resampling of some kind. However, it is not defined how to
resample. Possibly it should be as close as possible to an ideal resampling,
in which case that should be stated. Alternatively, it could be possible to
specify which kind of resampling to perform via an attribute: nearest,
linear, cubic, sinc, etc...

It also needs to be defined what should be done about folding when the net
result of the sample rates and playback rate is a downsampling, if anything.

I agree that a .resamplingType attribute could be defined, but holding off on that for now, since it's something which can later be added. In the mean-time, I think we should specify that the default is "linear". Does that seem ok?

[olivierthereaux]

Original comment by Philip Jägenstedt on W3C Bugzilla. Wed, 17 Oct 2012 08:42:10 GMT

Yeah, that sounds OK.

[padenot]

I don't personally think that linear resampling by default is ok when dealing with sound, but I can see that sometimes you'd rather trade quality for speed (especially if you know exactly what you are playing).

Gecko and Blink/Webkit have been using different resampling technique for a while, now, and I haven't heard any complaints. Why not simply state that the resampling technique to use is up to the UA?

We could also, as noted previously, expose a resampling attribute on the AudioBufferSourceNode:

enum  ResamplingType {
  "speed",
  "quality"
}

partial interface AudioBufferSourceNode {
  attribute ResamplingType resamplingType;
}

As for explicitly stating the algorithm to use in the type ("linear", "sinc", etc...), while there is (kind of) one way to implement a linear resampler, high quality interpolation will often use a combination of techniques, so I'd rather not state the name of the algorithm explicitly, and let UA compete in this field.

[joeberkovitz]

This doesn't seem to be agreed yet. We can review in WG but I also think it's a reasonable option to leave till V.next as leaving this unresolved is rather similar to Paul's suggestion, which I like.

[joeberkovitz]

TPAC: Adopting Paul's suggestion of letting UAs compete. Closing.

[CWBudde]

I think Paul's suggestion is good to have at least a switch to choose between speed and quality. However, I think it should have at least 3 options: speed (for something like the poor linear resampling), normal (for some good resampler) and quality (for a high sophisticated sinc interpolator or something equal to that).

The latter can be important if you want to resample filter kernels from an offline context. The performance doesn't really matter here as the kernels typically are very short. However, the quality must be higher than for normal resamplers.

[rtoy]

On the other hand, if it's important enough for you to have quality, only you know what quality you really need so the only way to be sure is to do the resampling yourself, either offline or online.

Alternatively, vote for adding a sample rate option when constructing an AudioContext so you can specify the rate that matches kernel sampling rate. Of course, you're then at the mercy of whatever resampling the AudioContext needs to do to get the audio out to the device.

[CWBudde]

I don't mind implementing the resampling myself and that's what I need to do now anyway. But one reason why I picked JS for in the first place is because it already offers the WebAudio API out of the box and without the need to implement something by myself.

How can I vote for adding the sample rate option? At least in terms of development speed this would eliminate adding the offline renderer step for my application (for the first draft).

[haywirez]

Also related to a v2 issue, especially @padenot's old comment https://github.com/WebAudio/web-audio-api/issues/118#issuecomment-61623298