Webrtc-pc: how to use datachannel bufferedamountlow threshold callback?

[fippo]

spawning from https://bugs.chromium.org/p/chromium/issues/detail?id=878682

one interpretation of this is the following:

sendChannel.bufferedAmountLow = 8192
sendChannel.addEventListener('bufferedamountlow', listener);
sendChannel.send( ... 16384 bytes of data )

According to the spec text, sendChannel.bufferedAmount should go to 16384 and consequently cross to 0. This seems like a nice way to do things. But it seems we can't have nice things. Both Chrome and Firefox seem to hand this off to SCTP immediately, not increasing bufferedAmount until the sctp buffers are filled.

This requires a slightly different usage pattern, taken from here

var sendAllData = function() {
    // Try to queue up a bunch of data and back off when the channel starts to
    // fill up. We don't setTimeout after each send since this lowers our
    // throughput quite a bit (setTimeout(fn, 0) can take hundreds of milli-
    // seconds to execute).
    while (sendProgress.value < sendProgress.max) {
      if (sendChannel.bufferedAmount > bufferFullThreshold) {
        sendChannel.addEventListener('bufferedamountlow', sendAllData, {once: true});
        return;
      }
      sendProgress.value += chunkSize;
      sendChannel.send(stringToSendRepeatedly);
    }
  };

basically calling send() until the SCTP buffers fill up, the browser buffers start to fill and get drained.
Straightforward as well but we don't have an example showing this and the spec text prohibits it.

See https://bugs.chromium.org/p/chromium/issues/detail?id=496700 for the original chrome implementation bug, and https://bugzilla.mozilla.org/show_bug.cgi?id=1178091 for Firefox.
Note in particular @jesup's comment 'bufferedAmount can't currently include data buffered in the stack'.

@lgrahl is what jesup said still true?

[lgrahl]

Yes, what Randell said is still true but I don't think it matters. I made a suggestion in the Chromium issue how this can be fixed by browser vendors. For details, see my comment in the issue.

I don't see anything in the example you've provided that would work around the current bug. This is how it's actually supposed to work. Send until bufferedAmount >= myHighWaterMark. Pause. Continue sending once the event has been fired. Awkward but working as intended.

Btw, funny enough the datatransfer example is likely to suffer from bad throughput once this is fixed by Chrome and Firefox since SCTP will not have enough buffered data. I would suggest using a higher high water mark to ensure a couple of chunks are in the buffer.

[alvestrand]

We do have WPT tests that fail because browsers don't behave as specified, so there's test coverage.

Browsers need fixing.

[fippo]

Having a test that is fails and is ignored by everyone isn't really helpful. "send until bufferedAmount >= myHighWatermark" seems like a good strategy (bonus: works currently!) so maybe we should show that in an example?

[henbos]

A lot of these discussions are based on misunderstanding about what a browser is supposed to be doing, Chrome is buggy. We should fix that.

I don't know what this bug wants us to do? We should not add workaround high buffers in the spec to get around the fact that browsers are not implementing the spec.

Is there something that needs to be done for this issue if browsers were implemented correctly?

[lgrahl]

Perhaps we should add an example as I'm frequently told by people who use data channels that they don't understand how it's supposed to be used or ask why they should use it at all. Then they see exceptions when sending (or data channels being implicitly closed, looking at you Chrome) and do not associate it with the lack of a flow control.

[fippo]

even if browsers implement this correctly (and I disagree on this being a bug) having an example would avoid the confusion.

[henbos]

Chrome's implementation, likely the cause of all of this confusion, has been fixed. Reading the spec again it seems pretty straight forward: "When the bufferedAmount decreases from above this threshold to equal or below it, the bufferedamountlow event fires."

I'm closing this issue. Feel free to object, but if so let me know what you think the example should contain.

[lgrahl]

@henbos The implicit closing is still there in Chrome.

I believe an example would be good and, if desired, I could provide one. This is the most elegant solution I've been able to come up with so far: https://github.com/saltyrtc/saltyrtc-demo/blob/3280e7046f7ba1379b846a4960a3a1c356ce4381/web/scripts.js#L24:L64

One would then just do this in a loop:

await fcdc.ready;
fcdc.write(message);

[jerlam06]

@henbos The implicit closing is still there in Chrome.

I believe an example would be good and, if desired, I could provide one. This is the most elegant solution I've been able to come up with so far: https://github.com/saltyrtc/saltyrtc-demo/blob/3280e7046f7ba1379b846a4960a3a1c356ce4381/web/scripts.js#L24:L64

One would then just do this in a loop:

await fcdc.ready;
fcdc.write(message);

Thanks for sharing this! However, could you please be more specific on why you chose the following values for high/low threshold:
lowWaterMark = 262144, highWaterMark = 1048576

Shouldn't the highWaterMark value be closer to 16MB, which is the maximum bufferedAmount?

Thanks :)

[lgrahl]

That was more of an arbitrary decision on what works well in practice. Choosing these values is an optimisation between the amount of JS calls (and thus likely CPU usage), the amount of data buffered (and thus memory pressure) and the network throughput. The last one has always been the bottleneck in my testing, so 1 MiB did the job. Keep in mind that the browser also has a buffer and the SCTP stack also has a buffer that you just don't see. :slightly_smiling_face: