Lighthouse: Make page refresh optional, depending on config

[paulirish]

Let's say the user only wants to run the accessibility audits. We dont technically _need_ to reload the page.

And the user may have navigated the app to a specific state they want to test, or made some DOM tweaks and want to evaluate if they are working better.

These usecases seem valid, so I'm thinking we should figure out if an audit requires a page reload. We can then communicate to the user if we expect a reload or not.

Just guessing, but I think we already know this. Do any of the audit's requiredArtifacts have a gatherer with a beforePass? If not, no reload technically needed.

cc @alice

[patrickhulce]

This type of information would also help in the more general effort of minimizing passes (in this case 0 😄 ). Network records are a special case wrinkle that the gatherer may not have a before pass but still needs a page load. We might want to reflect that dependency more explicitly moving forward.

[wardpeet]

Perhaps we should remove the term passes from config and let our gatherers decide? or have an internal config that knows which gatherers need a page reload or which can load together but might be hard to maintain or not even doable.

[paulirish]

Including @minorninth's comments from #2286

One of my favorite features of the previous accessibility audit (via the extension) was being able to modify the site and then run the audit again.

Could you make the page refresh optional, or maybe even just skip refreshing automatically if you only select the accessibility audit?

[ribrdb]

I would also like this so you can run audits on LTI tools. These are launched from a form post, and refreshing the page causes an error.

[joshuapease]

This option would greatly help with testing accessibility on SPAs and any other sites with highly interactive elements.

[EnlightenMeee]

Does Lighthouse Support SPA? i have a SPA project and Page 1 and Page 2 have the same URL but whenever i run lighthouse it only audit Page 1.
Is there an option to audit Page 1 and then clicks a button to go to Page 2 then Audits Page 2?

[chaitanyavkulkarni]

Having the same question as above. Does Lighthouse support SPA project ?

[carmacleod]

Would love to see priority bumped up for this one. I used to recommend @alice's Accessibility Audit to my colleagues all the time (it had this great "fix my color" feature :slightly_smiling_face: ).

However, I have been unable to recommend the Lighthouse Accessibility Audit, because it does not work for our pages - it just Audits our "Loading..." screen every time.

In order for us to be able to use Lighthouse for Accessibility Audits, we need the page to _not_ be refreshed.
This would be true for anyone with slide-out panels, dialogs, menus, or any other dynamic feature that they want to audit for accessibility. Accessibility Audits need to happen on the current state of the page.

[tbelaire]

I would also benefit from this. I'm trying to test a compose widget, but refreshing the page closes them.

[Laurensvdw]

So... I assume it is still not possible to pass in a config for certain occassions? Like monitoring Accessibility. We require this feature.

@paulirish Please provide an update :)

[patrickhulce]

This is not on our roadmap for the near future. In the meantime, you can use the axe extension directly in DevTools and combine Lighthouse with puppeteer (discussion and examples in #3837) in CLI.

Lighthouse uses the axe library under the hood, so the axe extension should give you identical results as Lighthouse.

[carmacleod]

Can you leave it up to the developer? Nothing fancy - just a checkbox. Assume they know what they're doing/what they want. 😉

Mockup below. Alternatively, the checkbox label could say, "Reload page before audit (recommended)".

Here's a scenario for why I want this in DevTools and not as an extension. Say I'm standing at a colleague's desk looking at their new dialog. I say, "Does that text have enough contrast? Is the dialog properly marked up for accessibility? Let's check, real quick!" At this point, I would love to say, "Open DevTools, go to Audits, check Accessibility, uncheck Reload, Go". It has to be _quick_, and it has to be _easy_, and it has to _not_ be an extension, because we need permission to add those.

And it has to _work_. Currently, the Accessibility audit only works on the initial state of the page. For our apps, this is some "loading" page or some other skeleton of a page, which unfortunately means that the Accessibility Audit doesn't work at all for us.

I am in a position where I can recommend things. I used to recommend the Chrome DevTools Accessibility Developer Tools _all the time_. They were quick and easy and they fit right in to my developer colleagues' work flows. And since the page wasn't reloaded before testing, you could go anywhere, set up any state you wanted to test, and run on that state. For accessibility testing, this is really necessary.

Sorry if I sound frustrated. I just received an email from Lighthouse asking if I would recommend it to anyone, and the honest answer for accessibility testing is, "No, I wouldn't". (Not unless they have a static or almost-static page to test).

[patrickhulce]

Can you leave it up to the developer? Nothing fancy - just a checkbox.

This issue is tracking the work necessary to make that possible :) The entire internal workings of lighthouse make assumptions that we observe page load, so fixing this requires massive changes across many surfaces. We agree with you, and we'd like to make this possible. It's just an enormous undertaking.

[carmacleod]

Ok, thanks for the explanation. I was originally thinking that perhaps the phrase "depending on config" in the issue title was maybe holding things up, i.e. maybe some fancy config-dependent algorithm needed to be determined, instead of just letting the user untick a box. :)

Your explanation tells me that the issue is much deeper than that - thanks again. :)

At this point, I guess all I can do is humbly request that the priority be raised... :)

[connorjclark]

Nothing to add to what Patrick said, but I'll try to help by suggesting other approaches :)

As a stopgap solution, you could make a DevTools Snippet (cmd shift p "snippet") that uses axe-core (https://cdnjs.com/libraries/axe-core/3.5.3). With some minimal post-processing of axe.run().then(...) you could print out the violations (w/ associated nodes) in the console. Might work in a pinch, especially if you put the whole thing in a GitHub gist and make a shortlink to it for quick access on a colleague's machine.

It's too bad that the default DevTools experience doesn't work for you out of the box re: a11y. I suggest reaching out to the DevTools dev list: https://groups.google.com/forum/#!forum/google-chrome-developer-tools , explaining your workflow constraints. I'm not sure what a11y features they have cooking, but I would think shipping the Axe extension to CDT by default would be a huge win ... maybe they'd agree?