Wavesurfer.js: Does wavesurfer work with streaming audio?

[larnold]

Hi,

Does wavesurfer work with streaming audio or does it need to preload the entire mp3 file first?

Thanks,
Luke

[katspaugh]

Hi, @larnold

It needs to load the whole clip, yes. Try waveform.js by SoundCloud, I think they support streaming.

[pramodtech]

@katspaugh are you planning to have streaming sometime in bear future? I think that will be great value add. What is the feasibility of this? I was wondering if I could contribute to it.

[katspaugh]

Do you mean drawing as it streams/plays? That should be trivial, with media element source and real time analyzer node.

By the way, are you Indian? I'm in India right now :)

[pramodtech]

I mean similar to what soundcloud does, draw waveform using json data and then fill the colors as audio streams.

Yes, I am Indian :) which city are you in now? I lives in Bangalore.

[katspaugh]

I'm in Mumbai :)

Oh, this should possible. I didn't see the value of streaming + pre-rendered waveform at first, because SoundCloud already does that. But knowing that you do some processing with the audio (like filters) and SoundCloud isn't using WebAudio, we might fill this niche.

Technically, look at createMediaElementSource. We should add a method that takes an audio URL and an array of peaks to draw. Seeking has to be implemented in a different way we do with a buffer source. Namely, you just set the currentTime of a media element.

[pramodtech]

For now I got to settle down with some custom waveform (http://stackoverflow.com/a/19554141/287100) since we had to deliver it to client. I will take a look at your suggestions this weekend.

[lepinsk]

I'm also following the HTML5 fallback stuff. I see that there's a version of wavesurfer here that seems to have had support for it. Was this something that was working at some point and then removed? I've played around with the example project in that version and it doesn't seem to be fully functional...

[katspaugh]

@lepinsk it's not about falling back to media element completely (as it was the case with the removed version you refer to). We will still use WebAudio but with media element as a source.

[lepinsk]

Hi @katspaugh – thanks, that was actually what I was working on myself when I noticed the project here to do the same thing. I'll keep an eye on your progress here.

[ahdinosaur]

+1 for streaming support.

[d-H-]

another +1 for streaming support

[icecast]

another +1 for streaming support

[wolfbiter]

+1 as well. I see there is code for loadStream, @katspaugh is it working in master or is it still experimental?

[katspaugh]

It's in master, but it might be not exactly what people want. It just draws the waveform as the audio plays, no pre-rendering of any kind.

[Lumnock]

I'm currently trying to get the loadStream effect (drawing the waveform as the audio is played) but with the microphone plugin providing the audio input. Not an easy task it appears! Any help/solution would be appreciated @katspaugh!

[katspaugh]

@Lumnock, let's talk in a separate issue.

[chinshr]

+1 for streaming support

[jczimm]

+1 for streaming support

[voronianski]

+1

[chinshr]

What would it take to support streaming in ws?

[katspaugh]

What do you guys mean by streaming and why do you want it?

[jczimm]

By streaming I meant audio data piped to clients (think IceCast or SHOUTcast).

[katspaugh]

I understand what streaming means in terms of audio. I'm asking what you are expecting the waveform of a stream to look like and the purpose of it.

[jczimm]

The way I personally think it would look best is as a running waveform (scrolling), adding data to the right as it is received.

[thijstriemstra]

Would be cool to support it in the microphone plugin, I'll try to come up with something

[antferr]

+1 for streaming via RTSP or HTTP with live555MediaServer

[Niades]

I understand what streaming means in terms of audio. I'm asking what you are expecting the waveform of a stream to look like and the purpose of it.

@katspaugh I think it would be nice to show only a part of waveform - the one that is playing right now, and then animate it to the next part, and repeat forever. I imagine it looking like an oscillograph.
P.S. Multi-eyed kitties FTW

[katspaugh]

Hey @Niades! Nice avatar 😸

Realtime oscillograph-like waveform is actually built into Web Audio. Here's a tutorial: https://developer.mozilla.org/en-US/docs/Web/API/Web_Audio_API/Visualizations_with_Web_Audio_API

By the way, we do this in the microphone plug-in—is that the kind of visuals you mean?

[adi-do]

Hi @katspaugh
How is this gong? I need to start playing the audio before the complete download. Is this possible? I generate the waveform and save the rendered peaks into my DB so I won't have to decode the audio again. But I still can't display the waveform and play the audio until it is completely downloaded. That is a bad UX. Some consumers are used to skim the audios. By streaming they don't need to wait for five long downloads and they don't need large storage (think about mobiles and tablet users)

[mspae]

@free2shout You could use the web audio element as an audio source and pass the prerendered peaks so that the audio isn't downloaded completely before the peaks are being drawn (as seen here: https://wavesurfer-js.org/example/audio-element/)

I think there is a little confusion in this thread, so I'll describe three use cases described here as I see them.

1. Display a waveform of a large file before it has been downloaded ( @free2shout ) – This can be done by passing in the peaks data and using the media element (which allows you to play audio before completely downloading it)
2. Render an oscillograph-style in real time like in the microphone plugin
3. Render a waveform of a (potentially never ending) audio stream where the waveform is being dynamically extended while the audio is playing (either by using peaks data from the client while the audio is being decoded (not sure if this is possible) – or by requesting peaks data from the server and using the media element for actually playing of audio) – This, I think, is currently not possible, but would definitely be really nice.

[ZachSaucier]

I created a streamer-type of format using wavesurfer by sending a bunch of short audio clips from the server. On the frontend I received these pieces, aligned a bunch of wavesurfer elements inline, and listened to the stop event in order to know when to play the next section. Let me know if you need more detail.

[entonbiba]

@ZachSaucier it would be nice if you can create an example on codepen or something :)

[ZachSaucier]

I don't have a simplified (or working in the same capacity) version due to the official production server being too costly to keep running, but you can look at the JS of this older version for a start/structure of the implementation.

[r01010010]

+1 to show an incremental waveform as audio is being loaded WITHOUT having to change anything at the backend. It can be perfectly done with the audio api in my opinion, at least in chrome.