Lighthouse: PageSpeed Insight disregards a layout shift in CLS score

[sauravvarma]

My website has a huge banner dropping from the top which causes the entire website to shift dramatically, it can be seen in the frames recorded by PageSpeed Insights as well and yet PSI's Lab data tells us that there is 0 shift.

Provide the steps to reproduce

1. Run PageSpeed Insights on https://www.1mg.com

What is the current behavior?

CLS score is 0

What is the expected behavior?

CLS score should not be 0

Environment Information

• Affected Channels: PageSpeed Insights
• Lighthouse version: 6.0.0

[patrickhulce]

Thanks very much for filing @sauravvarma! I can reproduce this with the CLI and with DevTools as well. Looks like a Chromium issue where layout shift events are not being counted despite obvious layout shift. We'll file upstream 👍

Worth noting that if I apply 6x CPU throttling then it sometimes reports a layout shift, but this shouldn't be necessary when it's already obvious.

[sauravvarma]

Thank you @patrickhulce

It would be also great if we can have some sort of explanation of how layout shifts are being calculated for real users.

If you run PageSpeed Insights on https://www.1mg.com/otc/accu-chek-active-test-strip-otc139543

The lab tests show a very very small CLS score and we've tested the site on several devices with different aspect ratios and sizes and the result has been the same. Yet we see a variation in CLS score amongst users which is weird.

[patrickhulce]

Ah, hang on so the issue here is that the layout shift has had_recent_input set to true, even though in Lighthouse there has clearly been no input. CLS is defined to ignore shift after user input.

One possible work around is for Lighthouse to ignore this flag and recompute the cumulative score ourselves using the sum of individual shift scores.

[sauravvarma]

If that is the case? Why would the other url I just sent have a score greater than zero in PageSpeed Insights?

[patrickhulce]

Why would the other url I just sent have a score greater than zero in PageSpeed Insights?

This isn't a problem with all pages just ones that have input events immediately before the shift occurs. I filed https://bugs.chromium.org/p/chromium/issues/detail?id=1094974 to document the issue.

[connorjclark]

@tdresser We're noticing that for every page that emits trace events, in Lighthouse the first event always has had_recent_input set to true. Is it expected that the initial navigation is ignored for CLS?

The issue seems to be that the timer in the layout shift tracker is instantiated in the constructor, so is in the "active" state and ignores the first shifts. Perhaps it should be instantiated only on the first user input?

[anniesullie]

We'll take a look at the Chromium bug, thanks for the report!

[patrickhulce]

Turns out this was a result of our emulation being reflected "had_recent_input". With the upstream bug closed this is on us to fix.