Karma: New Launcher - Chrome Headless

[dignifiedquire]

@wesleycho mentioned he would like to work on this. So opening an issue for planning.

[maksimr]

@dignifiedquire reporter? Looks like it should be launcher :)

[wesleycho]

Should be launcher - from what I understand, we need to add support for xvfb (https://github.com/karma-runner/karma/issues/1320) in core before we add a new launcher, is this correct?

[dignifiedquire]

Yes it should be launcher :D

@wesleycho why do we need support for xvfb in core, as far as I understand there are chromium builds which are really headless:

• https://news.ycombinator.com/item?id=11839303
• https://bugs.chromium.org/p/chromium/issues/detail?id=546953#c54

[wesleycho]

Ah, was xvfb only for when there was not a headless version? I must've misunderstood the situation when researching this before.

Doing some more searching, it appears that the --headless parameter is going to be added to Chrome eventually - do we want to create a separate launcher for headless Chromium? How do we want to handle using the headless Chromium itself? Currently it looks like one has to build it - instructions here. Should we build it fresh as part of the npm install process?

[vivganes]

If we plan to build headless Chromium during npm install then it is going to increase the 'setup' time for people who don't want headless Chromium as well. Creating a separate launcher will not have this issue. Also it will be helpful to isolate headless from regular Chromium because headless chromium is new and not supported in all platforms yet.

[wesleycho]

IMO we should create a new launcher for headless Chrome.

I haven't gotten a chance to sit down and start hacking it out since I have been busy, but I should be able to start working on an implementation next week.

[EddyP23]

So what is the plan for this? Is there a repo or an issue I can follow to track progress on this?

[myitcv]

@EddyP23 in lieu of something official, you can create a custom launcher today, per this comment

[remcohaszing]

Chrome headless just works. See #2603 for an example configuration.

[maksimr]

@wesleycho am I right that we want to build Headless Chrome?

[pselden]

https://github.com/karma-runner/karma/issues/2489#issuecomment-286404590 -- is it supposed to still bring up a chrome window at launch (localhost:9876) when I run with the headless configuration?

[remcohaszing]

@pselden no, it is not.

The configuration I referenced works for me, my coworkers and in a Docker container.

Just for reference, here's the relevant part of the Dockerfile I use:

FROM node:7

# https://www.google.com/linuxrepositories/
RUN wget -q -O - https://dl.google.com/linux/linux_signing_key.pub | apt-key add -
RUN echo 'deb http://dl.google.com/linux/chrome/deb/ stable main' >> /etc/apt/sources.list

RUN apt-get update && apt-get install --no-install-recommends -y google-chrome-stable

Note that headless mode is only supported in Linux and you'll need a pretty recent Chrome version. Afaik Chromium from the Ubuntu PPAs doesn't support it yet for example.

[otroboe]

It seems to be on Chrome 59 (canary/dev).
https://www.chromestatus.com/features/5678767817097216

[jchull]

@remcohaszing posted example config that works on OS X with canary, worked with Karma 1.3

HeadlessChrome 0.0.0 (Mac OS X 10.11.6): Executed 218 of 302 (skipped 84) SUCCESS (4.473 secs / 4.047 secs)

browsers: ['ChromeCanaryHeadless'], customLaunchers: { ChromeCanaryHeadless: { base: 'ChromeCanary', flags: [ '--no-sandbox', // See https://chromium.googlesource.com/chromium/src/+/lkgr/headless/README.md '--headless', '--disable-gpu', // Without a remote debugging port, Google Chrome exits immediately. ' --remote-debugging-port=9222', ] } },

[wesleycho]

@dignifiedquire should we close this issue? It seems like Chrome launched with headless support on all platforms in a way that makes it simple for developers to enable it within the existing karma launcher.

[rogeriopvl]

+1 for "it doesn't need another launcher"

[rogeriopvl]

Also there's this PR for chrome headless support: https://github.com/karma-runner/karma-chrome-launcher/pull/111

[dignifiedquire]

closing due to the above explanations

[RobYed]

Thanks for pointing out how chrome runs headless with karma :+1:

On Windows 7 Chrome still launches an empty window, when running Karma. Is there any possibility to prevent opening that window? I already tried the options --new-window/min and --no-startup-window, without success.

[jvmccarthy]

@RobYed according to Getting Started with Headless Chrome, "Windows support is coming soon" for headless Chrome (with a link to Make headless work on Windows).

[RobYed]

@jvmccarthy oh thanks! I must have overlooked that.

[jonathan-kosgei]

@remcohaszing Thanks for your comment. How do I set this up such that the browser exits once all the tests have passed?

[mgol]

@jonathan-kosgei This is unrelated to the issue. Configuration is documented at https://karma-runner.github.io/1.0/config/configuration-file.html, look for singleRun.

[bennycode]

@jchull Thank you very much for sharing your configuration with us! I just tested it with Chrome v59 and it seems to work. 😃 However there is still a blank page that spins up on a Karma test run. Can this page be suppressed?

Screenshot:

[csvan]

Always very eager to know how to lose the blank window.

[jchull]

@bennyn @csvan The headless feature is mainly needed for automation environments, like a linux build machine with no UI. I get no window on OS X, and it works on headless linux.
Here is the chromium bug for windows support: https://bugs.chromium.org/p/chromium/issues/detail?id=686608

[mgol]

Windows support landed in Chrome 60, not 59 as it did for macOS & Linux. You need to wait 6 more weeks to have a headless version officially supported on Windows.

[bennycode]

Ok, thanks for all your replies.

In most of my projects I am using Travis CI to run my test suites with a Karma test runner. When running Chrome headless on a Linux build machine of Travis CI, then there is no longer any need for the following config, am I right?

before_install:
  - export CHROME_BIN=/usr/bin/google-chrome
  - export DISPLAY=:99.0
  - sh -e /etc/init.d/xvfb start

Which means that using xvfb to run tests will become obsolete with a headless Chrome.

[Georgegriff]

I've been playing with the Chome 60 Beta, with the following configuration:
customLaunchers: { ChromeUK: { base: 'Chrome', flags: ['--lang=EN-GB', '--headless', '--disable-gpu', '--remote-debugging-port=9222'] } }

The tests that I have that do some Locale testing no longer work when running headlessly, has anyone else encountered this issue?

[ghusse]

Hello there.

This project might be helpful to create a headless launcher: https://github.com/GoogleChrome/puppeteer

[ashconnell]

@ghusse I wonder if Puppeteer can run out of the box? Will most likely be looking at this in the coming months.

[ghusse]

It seems that it can, (headless) Chrome is embedded in the lib

[layershifter]

I've recently made PR that adds intructions for Puppeteer, it's really simple :+1:

[karsunke]

Just for completion, you can start Chrome now just with 'ChromeHeadless': https://developers.google.com/web/updates/2017/06/headless-karma-mocha-chai

[bennycode]

Karma configuration to use headless Chrome:

    browsers: ['ChromeHeadless'],

    customLaunchers: {
      ChromeHeadless: {
        base: 'Chrome',
        flags: [
          '--disable-gpu',
          '--headless',
          '--no-sandbox',
          '--remote-debugging-port=9222',
        ],
      },
    }

Note: You will need to set a remote-debugging-port.

[karsunke]

@bennyn you dont need the customLauncher anymore. It works with 'ChromeHeadless' only.

[bennycode]

@karsunke: Indeed! Thanks for the hint. 👏
Source: https://github.com/karma-runner/karma-chrome-launcher/tree/v2.2.0#available-browsers