Wavesurfer.js: Releasing 2.0.0

[mspae]

There has been an extensive refactor of wavesurfer within the last half year which resulted in a version which includes a lot of new features (namely a dynamic plugin system) but also some breaking changes. We have decided to release it as a version two. Changes to either branch will be ported to each other (especially bug fixes of course).

This issue is intended as a place for general discussion and feedback regarding the v2 release.

Edit 1: Added change I forgot to mention: MultiCanvas is the default and only inbuilt renderer

The version is _mostly_ stable and functional. Due to the nature of the refactor (an integrated plugin system for adding, initialising and destroying of plugin instances) all plugins are somehow affected. Some plugins are still buggy when used with the dynamic plugin API. Conventional use cases (i.e. cases where you just want to create an instance with some plugins – no dynamic adding or removing – should work just fine) (Below in the plugin API examples, this means 1. should already work, 2. and 3. don't yet always work in all situations and for all plugins)

Installing the release

To get the current beta release (at the time of writing) do npm install [email protected]

You can find the beta release(s) on the releases page or by running npm show wavesurfer.js | grep beta

Developing

To work on the new branch checkout the next branch, do npm install and then npm start – The library directory is exposed under http://localhost:8080

Generate the documention by running npm run doc (see below) – Tests can be run by calling npm run test

There are still lots of things which need doing. We welcome and are thankful for any help we can get. Please see right at the bottom of this post how you can help us.

Changes

(See a full list of all merged PRs for v2)

• ES6 Syntax: Refactored all code to use the new ES6/ES2015 syntax (especially arrow functions, classes, spread operators etc.)
• New build system: Webpack is being used as a build system to compile the code. It can be imported like this:

// EITHER - accessing modules with <script> tags
var WaveSurfer = window.WaveSurfer;
var TimelinePlugin = window.WaveSurfer.timeline;
var MinimapPlugin = window.WaveSurfer.minimap;

// OR - importing as es6 module
import WaveSurfer from 'wavesurfer.js';
import TimelinePlugin from 'wavesurfer.js/dist/plugin/wavesurfer.timeline.min.js';
import MinimapPlugin from 'wavesurfer.js/dist/plugin/wavesurfer.minimap.min.js';

// OR - importing as require.js/commonjs modules
var WaveSurfer = require('wavesurfer.js');
var TimelinePlugin = require('wavesurfer.js/dist/plugin/wavesurfer.timeline.min.js');
var MinimapPlugin = require('wavesurfer.js/dist/plugin/wavesurfer.minimap.min.js');

• Documentation: Added loads of jsdoc style documentation tags and the esdoc documentation generator (the documentation needs to be also made public, currently it is generated into ./doc by calling npm run doc) – The idea being that it is easier to keep documentation up to date easier than having to always edit HTML code in the gh-pages branch.
• New plugin API: Previously all plugins, backends and drawers relied on the window.WaveSurfer global variable. This made it difficult to work with module bundlers (You had to expose the WaveSurfer object, which means it's not really modular) – To fix this plugins now all follow a common format which is used by wavesurfer core to register the plugins correctly. (see an example of a plugin class at the bottom of this post)

There are three ways to add a plugin:

1. Adding and initialising plugin immediately (this will do in 99% of the cases):

var wavesurfer = WaveSurfer.create({
    container: '#waveform',
    waveColor: 'violet',
    plugins: [
        TimelinePlugin.create({
            container: '#wave-timeline'
        })
    ]
});

1. Adding a plugin immediately and initialising it dynamically:

var wavesurfer = WaveSurfer.create({
    container: '#waveform',
    plugins: [
        TimelinePlugin.create({
            container: '#wave-timeline',
            deferInit: true // stop the plugin from initialising immediately
        })
    ]
});
wavesurfer.initPlugin('timeline');

1. Adding and initialising a plugin dynamically:

var wavesurfer = WaveSurfer.create({
    container: '#waveform',
    waveColor: 'violet'
});
// adding and initialising a plugin after initialisation
wavesurfer.addPlugin(TimelinePlugin.create({
    container: '#wave-timeline',
})).initPlugin('timeline');

Also a plugin can be destroyed like this:

wavesurfer.destroyPlugin('timeline');

• New HTML init script: The html-init.js was extended to allow more complicated initialisation options (including autoloading of plugin code) – see #946 – Example usage:

<wavesurfer
  data-url="../media/demo.wav"
  data-plugins="minimap,timeline"
  data-minimap-height="30"
  data-minimap-wave-color="#ddd"
  data-minimap-progress-color="#999"
  data-timeline-font-size="13px"
  data-timeline-container="#timeline"
>
</wavesurfer>
<div id="timeline"></div>

• MultiCanvas is the default renderer: It supports the functionality of the default renderer but is not constrained by maximum canvas sizes.

Plugin class format

All plugins now follow a common format and their initialisation is handled by wavesurfer core. It is no longer necessary to manually initialise them.

export default class MyAwesomePlugin {
    /**
     * MyAwesome plugin definition factory
     *
     * This function must be used to create a plugin definition which can be
     * used by wavesurfer to correctly instantiate the plugin.
     *
     * @param  {MyAwesomePluginParams} params parameters use to initialise the
     * plugin
     * @return {PluginDefinition} an object representing the plugin
     */
    static create(params) {
        return {
            name: 'myawesome',
            deferInit: params && params.deferInit ? params.deferInit : false,
            params: params,
            staticProps: {
                staticMethod() {
                    // this will be added to the wavesurfer instance and can then be called
                    // wavesurfer.staticMethod();
                }
            },
            instance: MyAwesomePlugin
        };
    }

    constructor(params, ws) {
      // instantiate the plugin
    }

    init() {
      // start doing something
    }

    destroy() {
      // stop doing something
    }
}

Things left to do!

• Wavesurfer.js (including plugins) is a fairly complex piece of software, that's why it's very important to have lots of people test it and file bugs!
• Although the core library has been refactored and documented very thoroughly the changes in the plugin code have been very minimal. They still lack jsdoc style comments* and can still be optimised/need to be debugged for use by the dynamic plugin API (*This would be very useful since that would make them appear in the generated documentation.)
• There are still some memory leaks (which is also true of the current version) – identifying and fixing them would make the whole library stabler.
• In general a lot of issues are documentation issues in the sense that they crop up because stuff is undocumented. Documenting is a proactive way of stopping those issues from every cropping up. Correcting and writing documentation, writing tutorials and how tos and so on would be very cool!

[katspaugh]

@mspae I've force-pushed next into the master branch, so that we can accept PRs into the next branch by default. Should've done it long ago.

How do we go about backporting the PRs that got merged into the old master?

_Edit_: apparently, this diff can serve as a rough overview of things that need to be merged into the next branch.

[mspae]

Hey, sorry for the long absence. Unfortunately I'm pretty busy with work right now, so I don't have that much time to commit to working on wavesurfer core fully at this time.

I think concerning the unported changes (= stuff from es5 which isn't in es6 yet) there is nothing for it but to grind them down until we are up to date. But maybe we need to organise this in a clever way to maximise the effect of our time.

I just looked through the diff you mentioned and I think the commits can be categorised like this:

• Stuff that has already been ported but was put into seperate commits/stuff that isn't relevant for es6 branch
• New plugin & features: namely the elan-wave-segment plugin
• Bugs and optimisations: Basically everything that isn't covered by the above. I think these are the most pressing but luckily they are also the easiest to port. As far as I can tell they're mostly one-liners and none need to be adjusted for the API changes in the es6 branch.

Then there is of course the question of open PRs – I think bug reports specific to the es6 branch have not been flooding in, so I'm assuming that either people are not using it that much or that it is largely stable.

I have the feeling that there is still quite a bit of friction involved in maintaining this library – which kind of makes it difficult to do small bits of work on it because it is necessary to think of lots of things at the same time. Maybe it would be a good idea to automate more of the process:

• Documentation deployment to the website (I know I keep saying this and don't come up with a solution myself :yum: )
• Changelog generation (Maybe we could use something like commitizen to make this smoother)
• Also I know this might be controversial but we could adopt the policy of not merging PRs with new features if there aren't any tests included

[mspae]

First PR for the first part is out – I checked approximately half of the commits that were different, here is the list

This list includes all commits that are different that are later than "Commits on Mar 12, 2017"

• :white_circle: https://github.com/katspaugh/wavesurfer.js/commit/abc69a401001bf213ea8935ee9a5b1241d77aa88 → redundant
• :black_circle: 64bd48b242cb54532dbc1f9d6aaf58e605e58669 → merged
• :black_circle: a1ff8ea98565cc83797a4721698ba78d59cd7dc4 → merged
• :black_circle: 0bc87cbf5cf04672cb9173abdd0c200f1df96f23 → merged
• :black_circle: 1d92bfacac08dfc431ae4978babc6baac11b3821 → merged
• :white_circle: 1f320ab422bb03df7871f9bab2c184d1da901ea7 → redundant
• :white_circle: ab65f1182996a316d2fbe4b51dd73095027b2d33 → redundant
• :black_circle: 0552da8adc36a7104fe93808d1dbe1ff285a7a37 → merged
• :black_circle: d9f3394a808cb127c8ae07aed655a2ffc94c2397 → merged
• :black_circle: 29fd88db307d8f322ff4bd52ca578f8d1cb96203 → merged
• :black_circle: 7fd5a6d0f1686f4c5f171f6e48e0d54e784bbe19 → merged
• :white_circle: 828a43ed171db221a96e6ec2fd67fb4830c46259 → redundant
• :black_circle: dcd8b55baaa0b506c4169ed616453d8a00f22fef → merged
• :white_circle: bc57ec8c9197fcbc003b03a076721e32fac25609 → redundant
• :black_circle: 77783081cc62c27afdf2eca1f1dc6623e46cc08c → merged
• :white_circle: 7d8c1032ad46f6fe1b4d7d3f0d27aa2a68634374 → redundant
• :black_circle: 23b54a15fe73462963522c5bdc777e4d7328698a → merged
• :black_circle: 0066f2cd03dd8843516b2d278c4699007b5c0563 → merged
• :white_circle: b65d9e9adff7bced2c195c6c0c74bc92a5a2c047 → redundant
• :black_circle: ffef86e7c0a78a1b736bdba9a1058abe54910ba7 → merged
• :black_circle: 01c851a40e1df7dc4c2a5c3e2a46f2944216c825 → merged
• :black_circle: a761eb34285a89cede9bb082e2f016c3abd5b9da → merged
• :white_circle: 6699076ddbb343cd5e98d010d957c11df2779180 → redundant
• :red_circle: 9942a2378e979b76fe5d000110e5371ac177205c → todo
• :black_circle: 6b4ae8125db5ed6ddda0afdcfa34adf7879d4544 → merged
• :red_circle: cc7b8613eb07e5b958751a9c939444fb24c942a1 → todo

[mspae]

Second PR is out (#1193). I checked the other half. The only unmerged ones (the red circles) are plugin changes. I also didn't merge all the changelog changes so I can simply copy the current changelog in one go.

• :black_circle: https://github.com/katspaugh/wavesurfer.js/commit/f6ce2c30cdd6968b88a0e7cf5aad96a31ee81484 → merged
• :black_circle: a8a6a6f37f3a5b4dde15797ba08a985e98508780 → merged
• :white_circle: 17ecdcbcb9dc1642560ccffd8870227f7a02c593 → redundant
• :white_circle: aa7f3c216967e7746d6fe066a8775cb14160bbe4 → redundant
• :black_circle: 029af950770a10801b0dbedb1fc39f5b6b326175 → merged
• :red_circle: 947b48afefce4a8c50b6a36aaca4724c0b301217 → todo
• :red_circle: 1ab84392d0eae278c50b56ee4c1842d205b43be2 → todo
• :white_circle: f586e8c9141d0eb97e4d7004dc42eadb42fec160 → redundant
• :white_circle: 1e0a01cc238653da6d1adf28b7176c3988e45b69 → redundant
• :white_circle: 33be11a2d29e938cff626623bf4504dcd903c879 → redundant
• :red_circle: 51a92d26765cc1b3a9618be8415a2f13a5a79fd3 → todo
• :white_circle: b166f91f055e87d929572b10720c4d83ba4124ab → redundant
• :black_circle: d69fa6ec0b79eb3c8c3fcfe9ef35bb9a4764cd90 → merged
• :white_circle: f46df7794a153e0f7355db176389dd0b73343581 → redundant
• :white_circle: 1f0887eead9740b8612e66c0f759de7b6a684e9f → redundant
• :white_circle: 02b8a7d24ac0c021b89d9cc3887fc155be4f625e → redundant
• :white_circle: 401caac814935418fc747540fcd017a871e08b7a → redundant
• :white_circle: 9d5991364a0cf79891dfc43b8b30e23b7c3f83ae → redundant
• :white_circle: f1772202c9b986df2e951833d133f4fa8a8588b5 → redundant
• :white_circle: e633171350c006ef8404b9f50861ffa71200ad2a → redundant
• :white_circle: 8be87f7ffda9719d2817f0d88c729615b34bc976 → redundant
• :white_circle: f7c20acf4cb066d8071092cf618c996a996164da → redundant
• :white_circle: 6384d94ea76c7907dfa45394bc5c366f5cd7b1f3 → redundant
• :white_circle: 385578bf0220ad73532697e7d6674f30f0fb6e16 → redundant
• :white_circle: cf6e35b8236ad7c8a57c576cd359fdb46f644485 → redundant
• :white_circle: 8666da3d0ee28f7290530f340fd785c97ccdcc9b → redundant
• :white_circle: f185d97c825a92bfb69be80091587a9a19f66594 → redundant
• :white_circle: 5cf96c08295e05d46b463c22aa5813e9051fd735 → redundant
• :white_circle: f9426cb01a106a141413f929779ef540c9ecd1b4 → redundant
• :white_circle: 3d908c57766740f09be683dd26a279b1601e5240 → redundant
• :white_circle: b0c0a8bac29cd5dcf50d92fad7f6641e4f7dc5ef → redundant
• :white_circle: 270f2b2f0f6754c618aa0d0c5851f8cf20167c31 → redundant
• :black_circle: 210f93fd35b65beb8907c6706adde60cb1e4a822 → merged
• :white_circle: 0699fc7b017c9aab41595b676a79647bf5ee7953 → redundant
• :black_circle: bf370ed74344dc9e6b8f5fe983fb7f5fd0ba2df6 → merged
• :white_circle: 4ee2b2986b6d3ff824e390c24889fafa64967b37 → redundant
• :white_circle: 808030b2c372818d41198456e3833c7686afbe19 → redundant

[thijstriemstra]

Updated one of my plugins and wasn't too much work: https://github.com/collab-project/videojs-wavesurfer/compare/wavesurfer2

When can we expect a next release?

[katspaugh]

How about this weekend? Version 2.0.0 without the beta, right?

[thijstriemstra]

A beta 2 would not hurt (or RC1).

[thijstriemstra]

@mspae can you roll a new release (beta2) for the current master? I have some things I want to test.

[mspae]

@thijstriemstra I can only tag, I cannot publish to NPM. AFAIK only @katspaugh can do that.

[thijstriemstra]

Ping @katspaugh

[katspaugh]

Tagged & published!

[thijstriemstra]

Awesome thanks!

[thijstriemstra]

Well it (still) works fine, so I'm +1 on a final release this week!

[thijstriemstra]

@mspae @katspaugh can we set a release date for 2.0.0? 1st of october? tomorrow? I'll help out where I can.

[mspae]

So I'm in two minds about this: On the one hand staying in beta makes it clear that there may be bugs we don't know about, on the other hand this is really also the case for a proper release. Stability is subjective and relative :smile_cat:

Also there is the matter of semver: releasing 2.0.0 means we can't break backward compatibility until we release 3.0.0 so we need to be really sure that this won't happen before we make the final step and go out of beta.

In light of this I would propose we do this:

• Merge #1210 and release as beta 3.
• Wait until we are sure there is no major breakage. During this time …
  
  
  
  • ~Implement commit hook of prettify & add eslint config to check the formatting to enforce a formatting standard (You can see the same thing done in https://github.com/mspae/react-wavesurfer)~ → https://github.com/katspaugh/wavesurfer.js/pull/1216
  
  
  • add visual testing for timeline, regions and minimap plugins
  
  
  • also add "normal" tests for those plugins
  
  
  • ... then refactor regions (Also adding some of the fixes & features from https://github.com/katspaugh/wavesurfer.js/pull/1103)
  
  
  • (add browserstack CI config to automate visual regression testing)
  
  
• Then IMHO we can release 2.0.0 because regions plugin refactor shouldn't have caused any breakage we don't know about because of all the tests which are in place (hopefully) by that time.

As a timeplan proposal: I think we should do the first step (see above) ASAP then aim to release 2.0.0 by the 10th of october.

I'm also working on a major overhaul of the website (more docs, more consistent design, bootstrap v4 etc) – it will also fix #1090 – I'm about 60% of the way and want to have this finished by the 2.0.0 release.

So that would be my proposal. This is of course open to discussion. I would be really happy for help with any of the above.

UPDATE: Updated the prettify point with the PR for that.

[thijstriemstra]

1210 changes the way ready event works and im -1 on these changes. Ill give feedback there. Secondly, this has been in beta since march and without a new release this project seems dead.

[thijstriemstra]

Also, freezing API till v3.0 and all regtession testing seems way too strict. Its useful but it changes the dynamic of this project where before it was loosely maintained and released as often as possible. Having all this testing also requires reviewers and we don't have that manpower or userbase to help out from my experience with this project for last couple of years.

[mspae]

I totally agree with the "we need more releases" position.

I'm not proposing we stop adding features until 3.0.0, but to follow semantical versioning we can't break backward compatibility between releases in a major version. And for this purpose I think it is essential to have testing in place to make sure new features don't break existing functionality.

I don't quite see testing conflicting with loose maintenance. In my view it is absolutely necessary to be able to quickly add features and merge PRs, the alternative would be to either manually test any new code (which IMHO conflicts with the "loosely maintained" concept) or to simply hope nothing breaks (which means there can never be any reasonable amount of confidence in the stability of releases)

So yes, adding and maintaining tests is overhead which doesn't add features. I see that. But on the other hand it allows us to delegate a lot of the tedious part of the reviewing process (testing functional stability) to the CI system. This frees contributor resources which can be used for architectural discussion, documentation, support and new features. So I think testing actually saves a lot of time in the long run.

[mspae]

I understand your argument regarding the breaking changes in #1210 – It does delay the potential 2.0.0 release date because we need to make sure breakage does not occur. But for the reasons stated above I think it's worth it. In addition to the testing itself and the changes of the events the PR also contains a lot of fixes for bugs which were never reported because they only appear in certain param combinations and interactions. Having this kind of testing prevents these bugs from cropping up unnoticed in future.

What I can offer as a compromise:

• I close the PR
• we release v2
• I open PRs for the rendering fixes (see above) and the visual regression testing without changing the way events are handled. (Some of the fixes rely on the consistent waveform-ready/ready event firing, but we can make it work somehow)
• We release those as patch versions of 2 because they don't break anything

This way we can release immediately. What do you think?

[thijstriemstra]

@mspae What's the status now? I think we should really release this asap (before end of year) because I feel it's damaging the project that there hasn't been a new release in such a long time. @katspaugh thoughts?

[mspae]

@thijstriemstra That's what I meant in https://github.com/katspaugh/wavesurfer.js/issues/1092#issuecomment-333311162 – we can release now as far as I'm concerned. But I don't have publishing rights, so it's not up to me.

[thijstriemstra]

@katspaugh seems to be missing in action :fearful:

[katspaugh]

@thijstriemstra sorry, missed your previous message.
Publishing!

Edit: I've removed the beta postfix and published the package (see https://github.com/katspaugh/wavesurfer.js/commit/4a93027e15ab2cf35e017515f6cb7c0f3c84aa55).

Congratulations and thanks a lot!

[thijstriemstra]

Thanks! 2.0.0 is now available on https://www.npmjs.com/package/wavesurfer.js, on to next release!

[thijstriemstra]

@katspaugh unfortunately it seems something went wrong with the release, the dist folder is missing from the npm package:

$ ls node_modules/wavesurfer.js/
CHANGES.md  LICENSE  package.json  README.md  src

[thijstriemstra]

People starting to notice: https://github.com/katspaugh/wavesurfer.js/issues/1273

[thijstriemstra]

I opened #1274 that will build the dist folder when you run npm publish, can you take a look @katspaugh and release a new 2.0.1?