Howler.js: Play audio while updating buffer over time [WebRTC]

[AhmedBHameed]

Hi, I have the following scenario which i want to stream MP3 from server using socket.io.
The problem i want to play the audio while updated chunks comes overtime from the socket.

I'm not sure where to look exactly but i will show you two cases of mine:
First:

state = {
    isFirstLoaded: false,
    blob: null,
    isBlobed: false
};
componentDidMount() {
    let bufferStack = new ArrayBuffer(0);
    socket.on("streamInfo", ({ id, sizeInBytes }) => {
      console.log(id, sizeInBytes);
    });
    socket.on("streamStart", arrayBuffer => {
      const { blob, isFirstLoaded, isBlobed } = this.state;
      if (!isFirstLoaded) {
        bufferStack = arrayBuffer;
        this.setState({ isFirstLoaded: true });
      } else {
        bufferStack = this.appendBuffer(bufferStack, arrayBuffer);
        this.setState({
          blob: new Blob([bufferStack])
        });
        console.log(blob);
        if (!isBlobed) {
          this.setState({ isBlobed: true });
          this.playTrack(); // What sould this function be here ????
        }
      }
    });

    socket.emit("streamStart", "track2.mp3");
    socket.on("disconnect", function() {});
}

Second case which i was successfully playing the track but with stuttering gap between each chunk which i saw one person referring about garbage collector issue but still not sure!!

class AudioAPI {
    constructor()  {
      this.context = new (window.AudioContext || 
                          window.webkitAudioContext || 
                          window.mozAudioContext || 
                          window.oAudioContext || 
                          window.msAudioContext);
      this.scriptNode = this.context.createScriptProcessor(4096, 1, 1);
      if(!this.context) {
        console.error("Browser dose not support audio context API!");
      }
      this.audioStack = null;
      this.nextTime = 0;
      this.isSchedualed = false;
    }

    scheduleBuffers(audioBuffer) {
      var source    = this.context.createBufferSource();
      source.buffer = audioBuffer;
      source.connect(this.context.destination);
      if (this.nextTime == 0)
        this.nextTime = this.context.currentTime - 0.01;  /// add 50ms latency to work well across systems - tune this if you like
      source.start(this.nextTime) ;
      this.nextTime += source.buffer.duration;
      this.isSchedualed = false;
    }

    decode(arrayBuffer) {
      this.context.decodeAudioData(arrayBuffer, (audioBuffer) => {
        this.scheduleBuffers(audioBuffer);
      },
      (err) => {
          console.log("err(decodeAudioData): "+err);
      });
    }
  };

  const audio = new AudioAPI();

  var socket = io(`#{env == "production" ? "http://localhost:5000" : "http://localhost:5000"}`), clientId;
  socket.on("streamInfo", ({id, sizeInBytes}) => {
    clientId = id
    console.log(sizeInBytes);
  });
  socket.on("streamStart", (arrayBuffer) => {
    audio.decode(arrayBuffer);

  });
  socket.emit('streamStart', "track2.mp3");
  socket.on('disconnect', function(){});

In the second case i was able to play the sound with this stuttering gap. Is there a better way to use your library solving my issue ?

Sorry for being long.
Thank you.

[crazywire]

I don't understand why you wanna use socket.io to stream mp3 file. Howler supports streaming using HTML 5. If you wanna send chunks from the server, you can use the content-range header and the browser will do the heavy lifting.

[AhmedBHameed]

@crazywire Very good question indeed. HTTP request is not very robust with very high speed audio seeking. I noticed that when you stream with content-range and start dragging on seeking with fair speed, that operation will cause node server to loose tracking some requests during seeking and prevent from going to end life cycle of a request. While it is streaming, it will make the server keep several instance streaming for dropped request (in client side).

By using socket.io your have a very robust connection that will prevent your server from loosing client requests. And that is why i wanted to be played by socket.io.

BTW Thank you for your note here and hope you get the idea.

[crazywire]

If you are using Web Audio API, Howler will wait until the whole file is downloaded before loading the audio in buffer.

You can dowload the chunks your self and pass them to Howler as base64. You can then play/pause them by passing their ids. Howler ids are 0, 1, 2, 3 etc { first sound is 0}. Let me know if this works for you.

[AhmedBHameed]

@crazywire No, this is not a real time application. Imagine we have 100 MB file size, would you wait for 5 minutes to play it ??
And that is why i need a real time audio player.
As the example above, the audio will play correctly but with small stutter of sound. Hope you understand what i mean.

[ShaharMynd]

@AhmedBHameed did you manage to find any solution?

[AhmedBHameed]

@ShaharMynd, It is an old topic that took some time. I ended up with Media Source Extensions from google. This api solved my issue. But it still have a side effect which i could not control seeking bar. It might be some lack of knowledge from my side.
I recommend (Personal perspective) you to find another way in case of big audio streaming.

[falk-stefan]

@AhmedBHameed Were you able to fix the seeking bar issue with Media Source Extension? I was about to get started with howler.js but then I saw your post. I have the same issue: I need to be able to stream larger audio files without a huge delay for downloading the entire file.

[AhmedBHameed]

Hi @falk-stefan
Actually, I stopped working on my project. But still interested to know the solution so far.