Brave-browser: Devtools "Audit" (Lighthouse) feature causes browser to freeze / lock up

[xanzhu]

Test plan

See https://github.com/brave/brave-core/pull/4044

Description

Feature is freezing / locking up when trying to use it.

When trying to audit a website the tool will lockup on run. This lockup also occurs when attempting to cancel it. After another cancel the tool will close then unable to run audits until the browser is relaunched.

Steps to Reproduce

1. Visit any site which uses https or use localhost.
2. Open developer tools (Rightclick and select inspect or ctrl + shift + i)
3. Navigate to the audits tab, click on run audits

Actual result:

Brave

Expected result:

Chrome

Reproduces how often:

Easily reproduced

Brave version (brave://version info)

Brave | 0.59.34 Chromium: 72.0.3626.81 (Official Build) (64-bit)
-- | --
Revision | ac8b982e05014492d1bd7d317628a4f22a97ffa0-refs/branch-heads/3626@{#796}
OS | Windows

Reproducible on current release:

• Produces on all clients (Dev/Beta/Release)

Website problems only:

• Disabling Brave Shields has no effect
• Issue does not occur on the latest version of Chrome

Additional Information

• Changing any setting within the audits UI does nothing as the issue still occurs.
• Worked normally in previous version (58.21)

[rebron]

@BhuZha can you try with shields down?

[xanzhu]

@rebron

[llexan]

+1 Also experiencing this issue on the latest and dev releases with shields being up or down.

[MuhammedKpln]

+1 Same problem exists on my Elementary os juno.

[llexan]

As a quick-fix for others running into this issue, installing the Lighthouse plugin and using that seems to work well: https://chrome.google.com/webstore/detail/lighthouse/blipmdconlkpinefehnmjammfjpmpbjk/

[anpenava]

for me this issue is also present, lighthouse just stuck on 'Lighthouse is warming up...' part, on any website I try to run it

Brave just installed:
Version 0.62.51 Chromium: 73.0.3683.103 (Official Build) (64-bit)

[xanzhu]

@rebron Issue still occurring on all versions
Any idea what might be causing this?

[VMBindraban]

Still happening on:

Brave | 0.64.77 Chromium: 74.0.3729.169 (Official Build) (64-bit)
-- | --
Revision | 78e4f8db3ce38f6c26cf56eed7ae9b331fc67ada-refs/branch-heads/3729@{#1013}
OS | Mac OS X

[manrueda]

Looks like the problem is that devtools tries to get a file from chrome-devtools://devtools/remote/serve_file/@9a9aa15057b6b2cc0909bdcf638c0b65ecd516f2/audits2_worker/audits2_worker_module.js and that request never finish or even timeout.
I also discovered that other modules like "Performance" also make requests to this chrome-devtools://devtools/remote/serve_file/** path and all request hang.

I would like to dig deeper on this but don't know what piece handles the requests to that URL pattern.

[xanzhu]

Still present on both Windows & MacOS running latest versions and on all clients.

Brave | 0.68.131 Chromium: 76.0.3809.100 (Official Build) (64-bit)
-- | --
Revision | ed9d447d30203dc5069e540f05079e493fc1c132-refs/branch-heads/3809@{#990}
OS | Mac OS X

@bsclifton , @rebron

[cupcakearmy]

Issue here too.

Brave: Version 0.68.132 Chromium: 76.0.3809.132 (Official Build) (64-bit)
OS: macOS 10.14.6

[willstocks]

I don't know whether this is relevant, but I mention it because Pagespeed was rebuilt to use the Lighthouse framework... but whenever I've tried to run a Pagespeed test via Brave, it has never worked because of the below:

So I assume it's a similar issue for Audits?

[forusak]

@willstocks it is not same issue since pagespeed works with shields down, unfortunately it doesn't give as much informations as audit does :(

[bsclifton]

I believe this may be broken because we disable remote debugging- which is captured with https://github.com/brave/brave-browser/issues/5640

This was disabled in https://github.com/brave/brave-core/pull/790 - per the issue it fixes (https://github.com/brave/brave-browser/issues/1736):

The feature requires downloading files from Google servers.

At a minimum, the download should be blocked. Better if we can disable the entire feature altogether. Even better if we can remove the files from the build.

Comment left in the remote debugging issue (https://github.com/brave/brave-browser/issues/5640)... subscribe to that if you haven't already

[bsclifton]

Fixed! Will be in our next Nightly (1.3, which is due to ship Feb 4th). We may uplift to 1.2 (Dev, shipping Jan 6th) if there's enough support for it and if no problems are discovered 😄

[nohl]

This tool is pretty helpful while we wait:
https://web.dev/measure/

[btlechowski]

Audit feature still does not work on 1.2.x. Tested on 1.2.29 on Windows 7 x64 and Ubuntu 18.04.
Checked commit history and 1.2.29 contains the uplift.

Works fine on 1.3.67 and 1.4.25.

Tested using test plan from https://github.com/brave/brave-core/pull/4044 for this issue.

@bsclifton @kjozwiak Do we want to fix this for 1.2.x or do we move the milestone to 1.3.x? If we move the milestone, we should consider reverting the PR from 1.2.x.

[bsclifton]

Confirmed this doesn't work on 1.2 - will look into this...

[bsclifton]

Good find, @btlechowski! Fixed with https://github.com/brave/brave-core/commit/1d23d47341a934fa5972a8e6a75decdb44f51014 (patch deletion didn't happen with uplift, for some reason). Next build, this will be fixed. I've confirmed this works great locally using the 1.2.x branch 😄

[bsclifton]

Removing QA/Blocked as this is fixed in 1.2.30

@btlechowski can you please try testing again? Thanks! 😄👍

[btlechowski]

@bsclifton good job! It works!

Verification passed on

Brave | 1.2.30 Chromium: 79.0.3945.79 (Official Build) beta (64-bit)
-- | --
Revision | 29f75ce3f42b007bd80361b0dfcfee3a13ff90b8-refs/branch-heads/3945@{#916}
OS | Ubuntu 18.04 LTS

Verified using test plan from brave/brave-core#4044 for this issue.

Warning message when Remote Debugging is disabled:

Audit working:

Verified passed with

Brave | 1.2.31 Chromium: 79.0.3945.88 (Official Build) beta (64-bit)
-- | --
Revision | c2a58a36b9411c80829b4b154bfcab97e581f1f3-refs/branch-heads/3945@{#954}
OS | macOS Version 10.13.6 (Build 17G5019)

• Verified test plan from https://github.com/brave/brave-core/pull/4044

Saw this message in terminal when Remote Debugging disabled:

Saw this after running the audit:

Verification passed on

Brave | 1.2.37 Chromium: 79.0.3945.88 (Official Build) beta (64-bit)
-- | --
Revision | c2a58a36b9411c80829b4b154bfcab97e581f1f3-refs/branch-heads/3945@{#954}
OS | Windows 10 OS Version 1803 (Build 17134.1006)

• Verified test plan from https://github.com/brave/brave-core/pull/4044
• Saw this message in the terminal when Remote Debugging disabled:
  
• Encountered https://github.com/brave/brave-browser/issues/7534

[jonathansampson]

I just tested Beta (1.2.34), Developer (1.3.73), and Nightly (1.4.42) on Windows 10 1909 (18363.535). I wasn't able to get Lighthouse to work in any of them. Steps were as follows:

1. Navigate to brave.com
2. Right-click on page, and select Inspect
3. Switch to Audits tab
4. Select 'Desktop' as device
5. Accept all other defaults
6. Press 'Run audits'

The audit doesn't appear to begin, even if I leave the window alone for a while.

[btlechowski]

@jonathansampson, you need to enable Remote debugging in chrome://settings/privacy first.

[jonathansampson]

@btlechowski Tried that on Nightly (1.4.42) and got the following:

[btlechowski]

Works on

Brave | 1.4.42 Chromium: 79.0.3945.88 (Official Build) nightly (64-bit)
-- | --
Revision | c2a58a36b9411c80829b4b154bfcab97e581f1f3-refs/branch-heads/3945@{#954}
OS | Windows 7 Service Pack 1 (Build 7601.24530)

@jonathansampson Don't forget to restart the browser after enabling debugging

[bsclifton]

@jonathansampson please do try restarting the browser; it doesn't need to be restarted on macOS- might be my bad for missing on Windows

For sure, you will need to reload the page (with the setting enabled) though

[jonathansampson]

@bsclifton Unfortunately, restarting the browser offered no relief :(

Protocol error (Runtime.evaluate): Promise was collected

Channel: DevTools
Initial URL: https://brave.com/
Chrome Version: 79.0.3945.88
Stack Trace: Error: Protocol error (Runtime.evaluate): Promise was collected
    at Function.fromProtocolMessage (devtools://devtools/remote/serve_file/@c2a58a36b9411c80829b4b154bfcab97e581f1f3/audits_worker/audits_worker_module.js:1634:121)
    at eval (devtools://devtools/remote/serve_file/@c2a58a36b9411c80829b4b154bfcab97e581f1f3/audits_worker/audits_worker_module.js:1232:246)

[bsclifton]

@jonathansampson what happens if you try another page / website?

[jonathansampson]

@bsclifton Good call. Google and Reddit both worked as expected. Something odd with Brave's site. Thank you for the sanity check. I'll create an issue regarding the Brave case, and investigate further as time permits.

[RobMaskell]

Still hanging with the lighthouse warming up message for me on both sites tested, one localhost and one live on internet. Linux PoPOS 19.10. This is on latest version after OS restart.
Tried on MacOS, same result.

[bsclifton]

@RobMaskell if you visit brave://settings/privacy, you should see a new setting Remote debugging which you can enable. Can you toggle that and then try again? You may need to restart the browser after changing it

As this isn't obvious, I created https://github.com/brave/brave-browser/issues/7645 to track proxying this (attempting to make it safe) and enabling it by default

[RobMaskell]

That fixed it for the internet (non localhost) site in MacOS, but for the same site on POPOS! after toggling on remote debugging it now runs but the results page renders with no real info on it and the following message
There were issues affecting this run of Lighthouse: Something went wrong with recording the trace over your page load. Please run Lighthouse again. (NO_FCP)

[Warface]

@RobMaskell same for me. I've enabled Remote debugging but go all errors from Lighthouse. Still have to use Chrome to audit which is a bummer

[jcliftonmeek]

Same. Total freeze.

[bsclifton]

@jcliftonmeek and you had enabled Remote Debugging under brave://settings/privacy? (I believe it requires a restart of the browser). It should be working after that

[connorjclark]

Lighthouse/Chromium eng here... some FYIs

The Lighthouse worker code is tied with the remote debugging feature of DevTools because it is one of a few modules that Chromium DevTools does not ship directly with the binary, to reduce size. Other items include the emulation device images. So when a user first navigates to the Lighthouse panel and runs it, the DevTools frontend will try to download from the remote module server the file at a specific revision, defined by the Chromium binary. If it can't find that file for some reason, it tries again with a fallback revision, before finally failing.

Since the revision that Brave uses will never match what is on Chromium's devtools remote module server [1], the hardcoded fallback revision will always be used in Brave.

On May 30, 2019 we renamed the files for the Audits panel, but never updated the fallback revision, not realizing this broke the fallback mechanism.

In June 2020 we updated the fallback revision for the remote module code that runs Lighthouse:.

As such, probably Lighthouse didn't work in Brave between then and last month. Although reading this PR, I see it worked in Brave ~Dec 2019, so it seems I'm missing piece of the picture.

This shouldn't happen again, now that we are aware of the problem. If we ever rename or move these modules again, we'll know to update the fallback revision. And if Brave really does rely on the fallback revision, you'll be happy to know we plan to update that much more frequently now.

[1] Definitely an assumption, I don't know how Brave is built. If the version from this [header] (as built by Brave) (https://source.chromium.org/chromium/chromium/src/+/master:out/win-Debug/gen/build/util/webkit_version.h;l=9;drc=d32eefd88b404ffff436a30c092c7e015516ce44;bpv=1;bpt=0?originalUrl=https:%2F%2Fcs.chromium.org%2F) matches a real revision in Chromium (and thus the devtools remote code server), then my assumption is wrong.

[bsclifton]

@connorjclark we've since removed the Remote Debugging setting and ship with it enabled by default (although, through a proxy). I haven't seen any user reports of Lighthouse not working - in all of our testing it has worked great

We do pull in the full Chromium code (with history) when doing a Brave build - so the Chromium SHAs should resolve just fine - we basically have the Brave code (patches, etc) under src/brave (which directly links to brave-core) and try to limit the patches we make to Chromium code

How does the Lighthouse script use the SHA? Is there a target we can check for in our output dir? Thanks for reaching out, BTW 😄

[connorjclark]

@connorjclark we've since removed the Remote Debugging setting and ship with it enabled by default (although, through a proxy). I haven't seen any user reports of Lighthouse not working - in all of our testing it has worked great

Cool! This implies to me that the hash in webkit_version.h must be passing thru to Brave unaltered. Otherwise, the fallback revision 100% wouldn't have worked for the duration I mentioned, and you'd have seen user reports of Lighthouse failing.

How does the Lighthouse script use the SHA? Is there a target we can check for in our output dir?

Lighthouse doesn't use it directly, but the DevTools frontend code does. Some modules in the frontend are marked "remote" and get downloaded through the process I mentioned. Most modules are bundled with Chromium.

There's no output to check for really. In an offline processing step, we have a service that checks out each revision of Chromium, builds the devtools frontend, and for that revision upload the output files to the remote module server.

FYI If you don't care for this entire remote module feature for Lighthouse (read: increased binary size is preferable to going to a remote service, even if via a proxy), you could probably tweak the BUILD file to bundle the Lighthouse worker instead of making it a remote module. https://cs.chromium.org/chromium/src/third_party/devtools-frontend/src/BUILD.gn?q=f:devtools+BUILD&sq=package:chromium&dr=C&l=213

Thanks for reaching out, BTW 😄

NP! Having never used Brave before, I was simply curious if Lighthouse worked in it. Came across this PR and was happy to see ya'll figured it out!