Lighthouse-ci: Feature: Compare different URLs

[schlessera]

Right now, the UI for the LHCI server allows comparisons only for the same URL across different commits.

I'm currently working on the AMP for WordPress plugin, and we have a different use case.

We have different subsystems where we want to measure the impact, and for doing so we have URL arguments to toggle them on and off. This is done because LHCI server only allows for a single report per URL on a given commit hash, so using different URLs for different variations of the same version of the code makes sense.

Example URLs:

'http://localhost:8080/',
'http://localhost:8080/?amp=1&amp_optimizer=0&amp_ssr=0',
'http://localhost:8080/?amp=1&amp_optimizer=1&amp_ssr=0',
'http://localhost:8080/?amp=1&amp_optimizer=1&amp_ssr=1',

The UI on the LHCI server does not allow us to see diffs from one URL to the other (whether that'd be on the same commit or a different one). It only allows for comparisons in time (different commits for the same location), but we'd also need comparisons across locations

Sticking to the same commit in that case would already be enough for our purpose, in case making both dimensions flexible at the same time proves difficult in terms of UX.

[patrickhulce]

Thanks for filing @schlessera! I like this feature request as a generic run comparison view where any URL from any build could be compared with any other.

[amedina]

This is a great feature and it will play a key role on evaluating and tracking optimizations towards improving core vitals, specifically in our case for sites powered by the AMP plugin.

/cc @paulirish @egsweeny

[gauthampughaz]

@paulirish I am also in need of this feature for my use-case. Any timeline on when it would be available as part of the stable release?

[patrickhulce]

There is no timeline for anyone working on this in the near future. It's open source though so feel free open a PR ;)

[gauthampughaz]

@patrickhulce any development guide available for this to better understand the existing implementation?

[patrickhulce]

There's architecture documentation that's definitely a must, but nothing really specific to the server UI implementation. I'll add some notes on it to CONTRIBUTING.md 👍

[patrickhulce]

FYI some guidance is now in @gauthampughaz #359