Wavesurfer.js: Unable to load audio from video tag

[qbunt]

Hey, thank you for testing and contributing to wavesurfer.js!

Please make sure you can check all of these points below before opening an issue:

(You don't have to post this section)

• [X] I have checked the FAQ and it doesn't solve my problem.
• [X] I have checked the documentation and it doesn't solve my problem
• [X] I have searched for already open issues which desribe my problem.
• [X] The issue I'm having is related to and caused by wavesurfer.js, not by other software (which maybe packages and uses wavesurfer incorrectly) – In that case you should open the issue on the respective project pages.

Please make sure you provide the following information (if applicable):

Wavesurfer.js version(s):

• 3.0.0
• 3.1.0

Browser and operating system version(s):

Chrome 77.0.3865.90
macOS Mojave 10.14.6

Code needed to reproduce the issue:

https://codepen.io/qbunt/pen/GRKVBrd

let wavesurfer = WaveSurfer.create({
          container: document.querySelector('#waveform'),
          waveColor: '#A8DBA8',
          progressColor: '#3B8686',
          backend: 'MediaElement'
})
let video = document.querySelector('video')
wavesurfer.load(video)

Use behaviour needed to reproduce the issue:

I originally encountered this in Vue, but have since validated that it shows up in all contexts and is not related to Vue at all. When attempting to start WaveSurfer with a MediaElement backend, and load a video element as the audio source, as shown in the docs, I get the following error consistently in every framework, and back to 3.0.0:

Uncaught Error: fetch url missing
    at Object.t.default (fetch.js:118)
    at t.value (wavesurfer.js:1448)
    at t.value (wavesurfer.js:1398)
    at t.value (wavesurfer.js:1322)
    at pen.js:11

Either this is something incorrect in the documentation, or something in Chrome changed and the URL isn't landing in the fetch call due to an incorrect reference or similar.

Of course, if this is something obvious that I'm doing wrong, I'd be happy to update the docs if that is the issue. Thanks so much, great lib!

[aburai]

i don't see that the code from the docs work, the first parameter from load will be used as source
the url that is created is https://cdpn.io/boomboom/v2/[object%20HTMLVideoElement]

try wavesurfer.loadElt(mediaEl)

[qbunt]

I just see:

pen.js:10 Uncaught TypeError: wavesurfer.loadElt is not a function
    at pen.js:10

Am I missing something?

[qbunt]

@aburai for what it's worth, everything I've tried with audio feedback in the current version of chrome outputs this:
webaudio.js:76 The AudioContext was not allowed to start. It must be resumed (or created) after a user gesture on the page. https://goo.gl/7K7WLu
Possible there was a change in policy that is preventing playback.

[qbunt]

I have <source> as an extra element underneath video as the spec allows for. If I adhere to this to allow for multiple sources, then I get the above failure. If the src attribute is attached to the <video> tag itself, it does work correctly after the audio context is defined.

[aburai]

sorry it is wavesurfer.backend.loadElt
and you must use the promise from play to handle os restrictions
see https://developer.mozilla.org/en-US/docs/Web/API/HTMLMediaElement/play

when media has src everythings works well
the above problem was because you uses the video element and the load method
sets the element to src which could not work
loadElt skips this part and directly use the media element

[aburai]

i look deeper in the code and in the example, loadElt does not what you want
your play directs to loadMediaElement and this method already check if the argument
is a string (url) or a media element and uses backend.loadElt)

i think we must add support for source tags there

[qbunt]

@aburai Do you know off-hand if the spectrogram plugin will load video as an audio element properly? I have the waveform working, the spectrogram plugin is throwing an error because it would appear that the backend (of MediaElement type`) does not have a buffer attached to the instance.

wavesurfer.spectrogram.js?e59a:609 Uncaught TypeError: Cannot read property 'getChannelData' of null
    at SpectrogramPlugin.getFrequencies (wavesurfer.spectrogram.js?e59a:609)
    at SpectrogramPlugin.render (wavesurfer.spectrogram.js?e59a:574)
    at t.SpectrogramPlugin._onReady (wavesurfer.spectrogram.js?e59a:470)
    at n (wavesurfer.min.js?b9b7:6)
    at eval (wavesurfer.min.js?b9b7:6)
    at Array.forEach (<anonymous>)
    at t.value (wavesurfer.min.js?b9b7:6)
    at t.eval (wavesurfer.min.js?b9b7:6)
    at n (wavesurfer.min.js?b9b7:6)
    at eval (wavesurfer.min.js?b9b7:6)

Which leads to:

value: function getFrequencies(callback) {
      var fftSamples = this.fftSamples;
      var buffer = this.buffer = this.wavesurfer.backend.buffer;
      var channelOne = buffer.getChannelData(0);
      var bufferLength = buffer.length;
      var sampleRate = buffer.sampleRate;
      var frequencies = [];

After setting some breakpoints, it looks like buffer, this.buffer and this.wavesurfer.backend.buffer are all null.

[aburai]

sorry no, i'd have to take a closer look.

[davegravy]

@aburai Do you know off-hand if the spectrogram plugin will load video as an audio element properly? I have the waveform working, the spectrogram plugin is throwing an error because it would appear that the backend (of MediaElement type`) does not have a buffer attached to the instance.

wavesurfer.spectrogram.js?e59a:609 Uncaught TypeError: Cannot read property 'getChannelData' of null
    at SpectrogramPlugin.getFrequencies (wavesurfer.spectrogram.js?e59a:609)
    at SpectrogramPlugin.render (wavesurfer.spectrogram.js?e59a:574)
    at t.SpectrogramPlugin._onReady (wavesurfer.spectrogram.js?e59a:470)
    at n (wavesurfer.min.js?b9b7:6)
    at eval (wavesurfer.min.js?b9b7:6)
    at Array.forEach (<anonymous>)
    at t.value (wavesurfer.min.js?b9b7:6)
    at t.eval (wavesurfer.min.js?b9b7:6)
    at n (wavesurfer.min.js?b9b7:6)
    at eval (wavesurfer.min.js?b9b7:6)

Which leads to:

value: function getFrequencies(callback) {
      var fftSamples = this.fftSamples;
      var buffer = this.buffer = this.wavesurfer.backend.buffer;
      var channelOne = buffer.getChannelData(0);
      var bufferLength = buffer.length;
      var sampleRate = buffer.sampleRate;
      var frequencies = [];

After setting some breakpoints, it looks like buffer, this.buffer and this.wavesurfer.backend.buffer are all null.

I ran into this same issue

[ZhouYing-bit]

I have the same problem

[Kerinlin]

@aburai Do you know off-hand if the spectrogram plugin will load video as an audio element properly? I have the waveform working, the spectrogram plugin is throwing an error because it would appear that the backend (of MediaElement type`) does not have a buffer attached to the instance.

wavesurfer.spectrogram.js?e59a:609 Uncaught TypeError: Cannot read property 'getChannelData' of null
    at SpectrogramPlugin.getFrequencies (wavesurfer.spectrogram.js?e59a:609)
    at SpectrogramPlugin.render (wavesurfer.spectrogram.js?e59a:574)
    at t.SpectrogramPlugin._onReady (wavesurfer.spectrogram.js?e59a:470)
    at n (wavesurfer.min.js?b9b7:6)
    at eval (wavesurfer.min.js?b9b7:6)
    at Array.forEach (<anonymous>)
    at t.value (wavesurfer.min.js?b9b7:6)
    at t.eval (wavesurfer.min.js?b9b7:6)
    at n (wavesurfer.min.js?b9b7:6)
    at eval (wavesurfer.min.js?b9b7:6)

Which leads to:

value: function getFrequencies(callback) {
      var fftSamples = this.fftSamples;
      var buffer = this.buffer = this.wavesurfer.backend.buffer;
      var channelOne = buffer.getChannelData(0);
      var bufferLength = buffer.length;
      var sampleRate = buffer.sampleRate;
      var frequencies = [];

After setting some breakpoints, it looks like buffer, this.buffer and this.wavesurfer.backend.buffer are all null.

Do you know how to solve this problem?I have the same problem