Sinon: Discussion: Abandon PhantomJS

[mroderick]

Chrome 59 will have headless support

• https://developers.google.com/web/updates/2017/04/headless-chrome
• https://www.chromestatus.com/features/5678767817097216

The single maintainer of PhantomJS is stepping down as a consequence of this: https://groups.google.com/forum/#!topic/phantomjs/9aI5d-LDuNE

We should probably start thinking about what we want to replace PhantomJS with.

• headless chrome
• https://github.com/tmpvar/jsdom
• your suggestion here

[lucasfcosta]

IMO headless chrome is the most adequate substitution.

When using headless browsers to make tests we do not want to test whether or not our code is using the latest APIs, we just want to make sure it works on a browser environment so I can't see why we should use other headless platforms since Chrome is what most users have, has great support, great and active team and works extremely fine.

Anyway, I'd love to hear what other people have to say and make sure I'm not missing any detail involved in this discussion.

[fatso83]

Chrome, for reasons mentioned above.

[mantoni]

I agree with @lucasfcosta.

When I read about headless Chrome, I instantly wanted to add support for it into Mochify. My approach to replace it would be to create a new project with an API and functionality similar to phantomic. It would then be trivial to add it to Mochify.

Also, I don't want to be a bottleneck in the development there. If anyone here wants to help, please ping me and I'll add collaborators. I might also group the related projects in a new org. Suggestions are very welcome.

[mantoni]

I managed to put an initial, somewhat working Chromium based implementation into Mochify. You can find it here: https://github.com/mantoni/mochify.js/tree/chromium

Tests pass, it works with Sinon and some other projects I tested. It's quite a bit faster running the tests, but startup time seems not much better than PhantomJS. I'm pretty underwhelmed with the performance gain. On a large test suite like Sinon, it only save around 200 milliseconds.

I went with Puppeteer which provides everything needed.

There is one downside to this: If an asynchronous error is thrown, it's handled correctly by Mocha, but there is no stack trace and some error details are lost. This works fine with PhantomJS. I'm not sure where this originates. I did a couple of experiments but was unable to get a stack trace other than the internal one from Puppeteer.

Outstanding work:

• Specify external binary (see skipped test case).
• Prevent navigation or deal with navigation events somehow. Apparently there are no plans to support this in Puppeteer / headless chrome as an option. While this is fine for Sinon, it's maybe not for other projects.

Please play around with it and leave feedback. Simply link mochify locally into Sinon and run npm run test-headless.

[mantoni]

Pull request is here: https://github.com/mantoni/mochify.js/pull/161