Wavesurfer.js: Spectogram plugin cuts out half of the canvas

[adw1n]

I noticed it when working with 44.1kHz files (in your examples you are using 22k ones and it works just fine).
This is what I get:

this is the same file in audacity (notice the 0-11k range):

this is the same file in audacity with range 0-22k

I changed line 46 in wavesurfer-js/plugins/wavesurfer/spectogram.js (granted this breaks things for 22k files and most likely in other cases)
from: this.height = this.fftSamples/2;
to:
this.height = this.fftSamples;'
and I got as a result:

my code:

```

```

music file that I used: https://github.com/arirusso/d3-audio-spectrum/blob/master/public/media/sweep.mp3

[adw1n]

Please also check 8kHz files - canvas is at least twice the size it should be (it contains garbage data).

[katspaugh]

@fzyukio is there a parameter to choose 44 kHz vs 22 kHz?

[fzyukio]

@adw1n this.height = this.fftSamples/2 is correct. @katspaugh the frequency is associated with each audio file so it is not a parameter to chose. FFT returns a symmetric representation of a signal, half of all FFT bins is negative frequency, which is the mirror version of the other half. The spectrogram only contains positive frequencies, hence its width is half the number of FFT bins (fftSamples). So setting this.height = this.fftSamples is mathematically wrong. The problem if any lies elsewhere. I'll test your code and get back later.

[adw1n]

@fzyukio but you are setting heightFactor to 2 in drawSpectrum (i'm working with mono files) which makes the effective height this.fftSamples/4 (only this.fftSamples/4 values get drawn for each bucket).

[adw1n]

Could you also explain why wavesurfer.backend.buffer.sampleRate==44100 for all files (tested on 8k, 22k, 44k)? http://wavesurfer-js.org/example/spectrogram/ soxi output:

Input File     : 'demo.wav'
Channels       : 1
Sample Rate    : 22050
Precision      : 8-bit

[thijstriemstra]

is this still an issue?

[adw1n]

@thijstriemstra yup - http://public.adw1n.com/wavesurfer.js-issue-702/

[Eddy155]

@katspaugh a question may not be of the subject but with wavesurfer you can cut the song and some example

[thijstriemstra]

Could you also explain why wavesurfer.backend.buffer.sampleRate==44100

I think it grabs the browser default, which is 44100 afaik.

[thijstriemstra]

maybe we should use an external spectrogram library for this plugin instead, e.g. https://github.com/vlandham/spectrogramJS

[shershey]

My coworker came across this bug report when he searched around after we discovered that the spectrogram frequency content was mislabeled and half of what we expected it to be. To demonstrate what I was seeing, we replaced the demo.wav file with a file that contains a 4kHz square wav (wav file with 16kHz sampling rate) and this is what I saw:

Notice that the fundamental frequency is incorrectly labeled at 8kHz. Changing the heightFactor = 1 fixed things. I also specified an audioContext with "audioContext: new AudioContext({sampleRate: 16000})". This proved important when displaying waveforms with more frequency content if their sampleRate was low since the default audioContext seems to resample things pretty high (44.1 or 48kHz) and this results in a lot of black space at the top of the spectrogram. This is the result with the changes:

Notice that the original demo.wav file has a sample rate of 22050hz and yet the demo implies there is frequency content above nyquist.

The issue appears not to be with the FFT/spectrogram but with that heightFactor. Does anybody know why it was originally set to something other than 1?

In an ideal world the spectrogram plugin could determine the original sampling rate and so something with that information (such as create an audioContext with the same sampling rate) so that it doesn't resample and create that empty space, but I guess in the case of compressed audio this might be trickier.

Without understanding what the heightFactor is for, I wonder if it can be removed. I guess that would change the look of everyone's spectrograms if they're used to seeing only half of the spectrogram Thoughts? Alternatively, assuming heightFactor has a useful purpose, could we make it explicitly programmable in the Spectrogram's params?

[thijstriemstra]

I guess that would change the look of everyone's spectrograms if they're used to seeing only half of the spectrogram

we're about to release 3.0 and breaking changes can still be made (if reasonable of course).

[shershey]

In discussing this with a coworker, we realized a better solution might be:

1) Remove heightFactor.
2) Modify the spectrogram to generate an offlineAudioContext with a sampleRate provided in the spectrogram's params. (using a different offlineAudioContext would allow for different sampleRates between playback and generating the spectrogram).
3) Give this a smaller default of something like 16kHz (So that by default any audio above a 16kHz sample rate will not have blackspace at the top of the spectrogram).

I'm not sure how much extra work it would be to create a different offlineAudioContext for the spectrogram than for playback. I could look into it.

[thijstriemstra]

maybe using a 3rd-party plugin for the spectogram so we don't need to maintain that code, e.g. https://blog.freifunk.net/2017/06/26/choosing-spectrogram-visualization-library-javascript/

[shershey]

Using a well supported spectrogram library sounds like a good idea. My hacky fix for my use case will be to provide an audioContext with a sampleRate twice my actual sampleRate.

[thijstriemstra]

hopefully you can work on a PR for this, I'll happily review it and provide feedback.

[shershey]

With the realization that I can work around this issue by setting the AudioContext SampleRate it reduces my need for a fix. But I can spend some time on this to see how much work it would take me and, if it's reasonable, send out a PR.

[thijstriemstra]

@shershey maybe a fix would be properly documenting that behavior in the source code as well as the example?

[shershey]

@thijstriemstra what's the timeline for the 3.0 release?

[thijstriemstra]

don't know. when it's done it's done, see #1651

[maxammann]

I just tried to create a spectrogram of a 16000Hz audio file. The max frequency in the spectrogram was 44100.

I fixed this by using audioContext: new AudioContext({sampleRate: 16000}).