Axe-core: Browserless Axe?

[alistair-hmh]

Is it possible to use Axe without a browser? (headless or otherwise)

E.g: Axe in JSDOM.

[jeeyyy]

@alistair-hmh

It is possible to run axe in headless mode.

If you prefer to use puppeteer, you will have to inject the page object in puppeteer with axe using the addScriptTag, and axe.run via the evaluate function.

By jsdom, if you are referring to jest with the preset of jest-puppeteer, yes, that can be done too.

[alistair-hmh]

Thanks for the info @jkodu !

By JSDOM, I'm thinking true "browserless":

• No Phantom
• No Puppeteer
• No Selenium

[WilcoFiers]

@alistair-hmh Yes, axe-core can run on JSDOM. We've got an example of it here:

https://github.com/dequelabs/axe-core/tree/develop/doc/examples/jest_react

Jest automatically sets up a document and window before running, but if that's not how you're doing it, be sure to set them up before you include axe. You may also want to disable the color-contrast rule.

Running axe-core against JSDOM does come with some performance costs. Browsers are a lot better at building and querying the DOM than JSDOM is, but if you're not running it thousands of times, it won't make a big difference. We use it internally this way as well for a few projects.

[molant]

Also no need to use jest. In webhint we are using axe-core directly with jsdom and also from a VS Code extension so you can get feedback while you are typing your HTML.

[alistair-hmh]

Browsers are a lot better at building and querying the DOM than JSDOM is, but if you're not running it thousands of times, it won't make a big difference.

Good to know. Thank you @WilcoFiers

How does running Puppeteer stack up against parallelization? I would have figured that running JSDOM would have a lot less overhead and be easier to scale up across cores.

Do you have any insights on that?

[marcysutton]

One important thing to note about JSDOM is it still doesn't have all the necessary APIs for the current color contrast algorithm, such as document.createRange. That rule is typically turned off in JSDOM settings to prevent errors. https://github.com/jsdom/jsdom/issues/317

[alistair-hmh]

Is there an example of axe-core running with JSDOM anywhere? I could not see one here (apart from a single test).

The README seemed to indicate effort is being made to run as many checks as possible in JSDOM. Is that something that's still happening?

If it is tested, is there somewhere to see a list of tests that pass and fail in JSDOM vs. Browsers, like compatibility table of some kind?

[molant]

Is there an example of axe-core running with JSDOM anywhere? I could not see one here (apart from a single test).

This is what we are using to run any script on jsdom . In line 30 you will have to put the code of axe-core.

This is for the old jsdom api (prior to 10). For the new one you can see the PR for that file here.

[alistair-hmh]

That's very cool! Thanks, @molant

Is there a way to get performance metrics from axe-core on non-react pages? I.e.: how long it takes to test a page like https://jestjs.io/docs/en/api ? - or is Axe always React specific?

[WilcoFiers]

Closing the issue, no action necessary.

[alistair-hmh]

I hacked up HTML_CodeSniffer so it could run in JSDOM across cpu cores via child_process streams.

If you're interested in check that out here: https://github.com/f1lt3r/ha11y

:o: ha11y can run HTML_CodeSniffer against https://jestjs.io/docs/en/api.html 40 times in 27 seconds on my MacBook mid-2015.

I'm not sure how you'd spread this work across processes in Pupeteer. Maybe cluster with PM2 and batch things off? (Could be super easy and fast, but I've not tried.)

I am interested to try the spreading axe-core work across child_process streams in JSDOM and compare speeds, quality of results, etc. I'll let you know if I get that working.

[F1LT3R]

I have Axe-Core doing the same thing as ha11y/codesniffer. Axe-Core is a lot slower, but I guess it covers a lot more rules?

Running across 4 cores, Axe-Core It takes 9 mins to sniff that previous Jest API page 40 times. Compared to 27 seconds for CodeSniffer.

I wonder what the overlap in rule coverage is between these two libraries.

[mike-strauch]

Any tips on getting axe-core to run on AWS Lambda (with nodeJS 12)? I'm hung up on figuring out a way to provide a valid window object.