Lighthouse: Bug report: for v3.0.3 launched from DTs in Chrome v70 all main audit metrics are identical

[annamikulinska]

Provide the steps to reproduce

1. Simply run LH 3.0.3 from Canary Chrome (70) DevTools (simulated throttling) on www.wlw.de (this is where the issue was first observed), www.google.com or www.yahoo.com - so far the issue replicates at every page we've checked.

What is the current behavior?

All the main Performance audit metrics display the same time (First Contentful Paint, First Meaningful Paint, First CPU Idle, Time to Interactive)

What is the expected behavior?

Snapshot from audit run on www.wlw.de around the same time in Chrome 68 DevTools with LH version 3.0.0-beta (those are slightly different from 2.9.1 but still matched our expectations - we've read the documentation... ;) )

Audit run 2hours later in Chrome 68 as well but with an extension (LH 3.0.3)

Environment Information

• Affected Channels: DevTools, Chrome extension
• Lighthouse version: 3.0.3
• Node.js version: -
• Operating System: HighSierra 10.13.6
• Checked in Poland and in Germany, if it matters ;)

Related issues

[patrickhulce]

When there are no long tasks beyond first paint and the page reaches a steady state quickly, this is the expected (and greatly desired!) behavior. The definition of all those metrics is such that FCP is the earliest point they can occur, so all of them being equal to FCP just means that the page isn't doing anything else that's bad to delay the other metrics 🎉

The jumping around of the performance score for the same page is most likely a separate issue though. The best thing to do to diagnose what's going on here is to collect traces of your runs (you can open the trace from DevTools via the "View Trace" button or with the CLI using the --save-assets flag). There are lots of reasons for performance variation as you know from the docs ;) but finding out which one is hitting you requires a bit more information.

[annamikulinska]

@patrickhulce, thanks for the quick answer but I guess I didn't express my concern clearly enough:

1. Chrome Canary v70 + LH v3.0.3 = all the 4 metrics: FCP, FMP, FCPUI and TtI are exactly the same (regardless of the website I run the audit for)

2. Chrome v68 + LH v3.0.3 = metrics are diversified (as expected)

[ As for the sudden drop in the FCP/FMP results from ~2,5k to ~900... all I care about are the proportions... ;) I will simply need to find a way to bring the F-CPU-I and TtI closer to it :) ]

[patrickhulce]

Oooooh gotcha, yes thanks for clarifying!

This is the same root cause as #5832, a different Chrome change broke our performance metric computation. It's fixed by #5841 but will take a few days to show up in Chrome.

[annamikulinska]

oooh, ok! Cool, thanks! 😸 Looking forward to seeing it solved 😸