Axe-core: axe-puppeteer

[anoop-gupt]

I would like to contribute in axe-puppeteer repo

[WilcoFiers]

We've got a puppeteer example that should be landing soon. @stephenmathieson When you've got a few min, can you put up that PR?

[anoop-gupt]

sure..thanks..

[stephenmathieson]

Yep! I'll push something shortly 😄

[jeeyyy]

Closing issue as PR has been merged.

[mohanraj-r]

@stephenmathieson @WilcoFiers
The doc at axe-core/doc/examples/puppeteer at master · dequelabs/axe-core has been very useful. Thanks!

Btw, How do I pass in the options parameter for axe.Run() inside page.evaluateHandle()?

[stephenmathieson]

Hey @mohanraj-r! The short/sweet example added in #1018 doesn't allow you to provide custom options to axe.run(). To get this to work, you'll have to slightly modify it:

// Whatever options you need go here:
const options = {
  runOnly: {
    type: 'tag',
    values: ['wcag2a']
  }
}

const handle = await page.evaluateHandle(`
  ${axeCore.source}

  // Run axe with your options.
  axe.run(${JSON.stringify(options)})
`)

I do want to warn you that the Puppeteer example is missing many useful features that axe-webdriverjs supports. It's not quite "production ready" - we've just got it there as a reference for developers looking to build their own library.

[mohanraj-r]

Thank you @stephenmathieson

1. I tried your suggestion, but couldn't get it to work. E.g. I am trying to restrict result types to just violations and incomplete. But when I run it, I am getting inapplicable in output as well

const axeConfig = {
  runOnly: {
    type: "tag",
    values: ["wcag2a", "wcag2aa"]
  },
  resultTypes: ["violations", "incomplete"]
};

 const axeRun = await page.evaluateHandle(`
    ${axeCore.source} // Inject axe source code
    axe.run(${JSON.stringify(axeConfig)}) // Run axe
    `);

Output:

...
 incomplete:
   [ { id: 'color-contrast',
       impact: 'serious',
       tags: [Array],
       description:
        'Ensures the contrast between foreground and background colors meets WCAG 2 AA contrast ratio thresholds',
       help: 'Elements must have sufficient color contrast',
       helpUrl:
        'https://dequeuniversity.com/rules/axe/3.1/color-contrast?application=axeAPI',
       nodes: [Array] } ],
  inapplicable:
   [ { id: 'area-alt',
       impact: null,
       tags: [Array],
       description: 'Ensures <area> elements of image maps have alternate text',
       help: 'Active <area> elements must have alternate text',
       helpUrl:
        'https://dequeuniversity.com/rules/axe/3.1/area-alt?application=axeAPI',
       nodes: [] },
...

1. > Puppeteer example is missing many useful features that axe-webdriverjs supports

Is this documented in detail anywhere? I saw @WilcoFiers mention in one of the other issue about iframe limitation. It would be great to if the missing features could be documented - so that it can also be used as a checklist when new releases of puppeteer are out.

[mohanraj-r]

@stephenmathieson looks like the puppeteer 1.9.0 that was just release has added new API for frame. Not sure if this address the issues you were referring to.

Also puppeteer/api.md at v1.9.0 · GoogleChrome/puppeteer

An example of getting text from an iframe element:

[stephenmathieson]

I tried your suggestion, but couldn't get it to work. E.g. I am trying to restrict result types to just violations and incomplete. But when I run it, I am getting inapplicable in output as well

Odd. I'll take a look today. I've been able to pass options to axe-core this way, so this is very strange.

Is this documented in detail anywhere?

No, not in detail. However the README of the example does state that the example is not feature-complete. Some things I can think of off hand are:

• <iframe> support
• enabling/disabling rules
• including/excluding elements
• locale support
• ...probably more

I'd like to re-mention that the Puppeteer example is not really a Deque-supported product. It's just a starting point for you to see how it's possible to integrate axe-core into several different systems.

looks like the puppeteer 1.9.0 that was just release has added new API for frame

Nice, thanks for the heads up! However, the reason the example doesn't support frames is not due to missing APIs in Puppeteer; instead, it's due to the amount of time it would take to build a fully-featured product.

[mohanraj-r]

Thanks for the clarifications @stephenmathieson

Aren't the following part of the axe configure api ?

• enabling/disabling rules
• including/excluding elements

Can you help me understand why these are not supported in puppeteer? Are these limitations on puppeteer's end? Or is it work to be done in axe to get it working with puppeteer?

I really love the ease of use of puppeteer and not having to go through an abstraction API and another layer (webdriver). Would offer any help that I can to get axe working in puppeteer. But if puppeteer has limitations that are not easily overcome - then I will have to change course ..

Btw, are there other ways in which axe.run() could be run on the page?
Looked at puppeteer-accessibility/accessibility.js - but its not clear how/if they process axe config options.

[mohanraj-r]

Would the limitations apply even if I use the page.evaluateOnNewDocument() API ?

Adds a function which would be invoked in one of the following scenarios:

• whenever the page is navigated
• whenever the child frame is attached or navigated. In this case, the function is invoked in the context of the newly attached frame
  The function is invoked after the document was created but before any of its scripts were run.

[stephenmathieson]

Aren't the following part of the axe configure api ?

Yes. The methods exposed in axe-webdriverjs are just "sugar" on top of the configure API.

Can you help me understand why these are not supported in puppeteer?

The Puppeteer example is not a fully supported Deque product. It's only an example of how to use axe-core with Puppeteer. This may change in the future, but until then, we don't intend on adding feature parity with our other products.

Are these limitations on puppeteer's end?

No. These limitations are on Deque's end. We do not currently have plans to support another product.

Or is it work to be done in axe to get it working with puppeteer?

Somewhat - we'd have to spec out and build a proper axe-puppeteer project.

Btw, are there other ways in which axe.run() could be run on the page?

I'm not sure what you're asking here.

Looked at puppeteer-accessibility/accessibility.js - but its not clear how/if they process axe config options.

They pass in options on line 21. Puppeteer then passes them to the browser's scope on line 17. Finally, they're passed to axe-core at line 20.

Would the limitations apply even if I use the page.evaluateOnNewDocument() API ?

Yes. These are not limitations created by Puppeteer. The reason the features you're asking about do not exist is because we did not add them.

[mohanraj-r]

Thanks very much for the clarifications @stephenmathieson

I understand that puppeteer isn't an official axe product yet. Maybe I can help with that. But using axe-core lib on puppeteer should work as documented in the axe-core API doc - shouldn't it? I understand that axe-webdriverjs provides convenience functions for configuring and running axe. But shouldn't the same configuration be ideally accomplishable using config options on the axe-core API on puppeteer? Just want to make sure I am not missing something deeper here.

Btw, I couldn't get the config to work using webdriver / directly in chrome devtools. Hence created #1183

[stephenmathieson]

Yes, using axe-core in Puppeteer works as expected. However, our implementation of a Puppeteer-based driver for axe-core is not fully featured. It does not inject axe-core into frames (amongst other things).

[stephenmathieson]

Heads up! We now have an official axe-puppeteer library, which supports all of the features axe-webdriverjs does.

[anoop-gupt]

Thats awesome