Lighthouse: Doing Network throttling programmatically?

[mohanrohith]

I am working on project where I am running lighthouse on a set of urls and collecting performance metrics on each build over a CI process. I am running lighthouse programmatically making use of lighthouse and chrome-launcher node modules.

I am looking for a way to simulate network throttling programmatically.

I am aware of https://bugs.chromium.org/p/chromium/issues/detail?id=728451
and https://github.com/GoogleChrome/lighthouse/blob/master/lighthouse-core/lib/emulation.js

Somehow I am not able to stitch things together.

[patrickhulce]

Hey there @mohanrohith great to hear you're putting Lighthouse in CI! If you're using LH out-of-the-box then 3G network throttling should be enabled by default. If you're looking to provide your own custom network throttling settings, you can follow our advanced throttling guide to get more control over the settings.

Let us know if there's anything lacking in there that you found you needed to do!

[mohanrohith]

Thanks for the quick response.

I was trying to avoid external dependencies and was looking for options within lighthouse that looks like below

const LATENCY_FACTOR = 3.75;
const THROUGHPUT_FACTOR = 0.9;

const TARGET_LATENCY = 150; // 150ms
const TARGET_DOWNLOAD_THROUGHPUT = Math.floor(750 * 1024 / 8); // 750kbps
const TARGET_UPLOAD_THROUGHPUT = Math.floor(750 * 1024 / 8); // 750Kbps

const TYPICAL_MOBILE_THROTTLING_METRICS = {
  targetLatency: TARGET_LATENCY,
  latency: TARGET_LATENCY * LATENCY_FACTOR,
  targetDownloadThroughput: TARGET_DOWNLOAD_THROUGHPUT,
  downloadThroughput: TARGET_DOWNLOAD_THROUGHPUT * THROUGHPUT_FACTOR,
  targetUploadThroughput: TARGET_UPLOAD_THROUGHPUT,
  uploadThroughput: TARGET_UPLOAD_THROUGHPUT * THROUGHPUT_FACTOR,
  offline: false,
};

 const flags: Types.IAppConfig = {
  "url": "https://example.com",
  "flags": {
    logLevel: 'info',
    disableDeviceEmulation: true,
    enableNetworkThrottling: true,
    disableCpuThrottling: true,
    settings: {
      TYPICAL_MOBILE_THROTTLING_METRICS
    },
    chromeFlags: [
      '--headless',
      '--no-sandbox',
      '--user-agent=StandardUA Custom/website/41/website/Desktop'
    ]
  }};

launchChromeAndRunLighthouse(url: string) {
    const config: Types.IPerfConfig = BaseConfig.PERF_CONFIG;

    return chromeLauncher.launch(flags).then(chrome => {
      flags.port = chrome.port;
      return lighthouse(url, flags, config).then(results => {
        return chrome.kill().then(() => results);
      }).catch(err => {
        return chrome.kill().then(() => {
          throw err;
        }, console.error);
      });
    });
  }

Let me explore the comcast options. I am closing this issue

[ebidel]

What about using Puppeteer to launch chrome and set throttling settings? This flow seems to work but @patrickhulce or @paulirish should verify I'm not missing anything:

1. disable the throttling settings in Lighthouse.
2. Launch Chrome using Puppeteer and tell Lighthouse to reuse Puppeteer's chrome instance.
3. Tell lighthouse to programmatically load the page.
4. Watch for the page to open and set the emulation conditions:

const puppeteer = require('puppeteer');
const lighthouse = require('lighthouse');;
const {URL} = require('url');

(async() => {
const url = 'https://www.chromestatus.com/features';
const networkConditions = {
  offline: false,
  latency: 800,
  downloadThroughput: Math.floor(1.6 * 1024 * 1024 / 8), // 1.6Mbps
  uploadThroughput: Math.floor(750 * 1024 / 8) // 750Kbps
};

// Use Puppeteer to launch headless Chrome.
const browser = await puppeteer.launch({headless: true});

// Wait for Lighthouse to open url, then customize network conditions.
// Note: this will re-establish these conditions when LH reloads the page. Think that's ok....
browser.on('targetchanged', async target => {
  const page = await target.page();

  if (page && page.url() === url) {
    const client = await page.target().createCDPSession();
    await client.send('Network.emulateNetworkConditions', networkConditions);
  }
});

// Lighthouse will open URL. Puppeteer observes `targetchanged` and sets up network conditions.
// Possible race condition.
const lhr = await lighthouse(url, {
  port: (new URL(browser.wsEndpoint())).port,
  output: 'json',
  logLevel: 'info',
  disableNetworkThrottling: true,
  disableCpuThrottling: true,
  disableDeviceEmulation: true,
});

console.log(`Lighthouse score: ${lhr.score}`);

await browser.close();
})();

[patrickhulce]

@ebidel unfortunately it won't be that easy since LH explicitly resets the network throttling to no throttling if you set disableNetworkThrottling: true, #3837 has some discussion and tracks work on our side to play better with puppeteer by not resetting protocol state all the time

[ebidel]

@patrickhulce Right, but presumably the await client.send('Network.emulateNetworkConditions', networkConditions); in my script comes after LH does that. Do we much with emulation settings after the initial setup?

[patrickhulce]

Maybe I'm misunderstanding when targetchanged fires then, we reset on every pass not just during initial setup, but if targetchanged occurs on each navigate (which I suppose it should?) then yeah that should be good to go 👍

the more explicit recommendation in #3837 was a custom gatherer that sets throttling settings in beforePass, but either way playing with DevTools throttling on your own can be not what you expect since the latency is at the request level rather than packet level just FYI future readers :)

[ebidel]

Yep, fires on every nav.

[mohanrohith]

Yeah I modified my scripts to use puppeteer to launch LH using following bandwidth configurations.

const NETWORK = {
  "cable": {
    offline: false,
    latency: 28,
    downloadThroughput: 5000000,
    uploadThroughput: 1000000
  },
  "dsl": {
    offline: false,
    latency: 50,
    downloadThroughput: 1500000,
    uploadThroughput: 384000
  },
  "threegslow": {
    offline: false,
    latency: 400,
    downloadThroughput: 400000,
    uploadThroughput: 400000
  },
  "threeg": {
    offline: false,
    latency: 300,
    downloadThroughput: 1600000,
    uploadThroughput: 768000
  },
  "threegfast": {
    offline: false,
    latency: 170,
    downloadThroughput: 1600000,
    uploadThroughput: 768000
  },
  "fourg": {
    offline: false,
    latency: 150,
    downloadThroughput: 9000000,
    uploadThroughput: 9000000
  },
  "lte": {
    offline: false,
    latency: 70,
    downloadThroughput: 12000000,
    uploadThroughput: 12000000
  },
  "edge": {
    offline: false,
    latency: 840,
    downloadThroughput: 240000,
    uploadThroughput: 240000
  },
  "twog": {
    offline: false,
    latency: 800,
    downloadThroughput: 280000,
    uploadThroughput: 256000
  },
}

async launchChromeAndRunLighthouse(url: string) {
    const flags: Types.IChromeFlags = this.config.FLAGS;
    const config: Types.IPerfConfig = BaseConfig.PERF_CONFIG;

    const browser = await puppeteer.launch({
      headless: true,
      args: this.config.FLAGS.chromeFlags
    });

    browser.on('targetchanged', async target => {
      const page = await target.page();

      if (page && page.target().url() === url) {
        console.log(`Target changed to ${url}`);
        await page.target().createCDPSession()
          .then(client => {
            return client.send('Network.emulateNetworkConditions', BaseConfig.NETWORK[BaseConfig.CONNECTION_TYPE]);
          })
          .catch(err => console.error(err));
      }
    });


    flags.port = (new URL(browser.wsEndpoint())).port;
    return lighthouse(url, flags, config).then(results => {
      return browser.close().then(() => results);
    }).catch(err => {
      return browser.close().then(() => {
        throw err;
      }, console.error);
    });
  }

It seems to work as my dashboard shows similar to what I am expecting but dnt know how to validate it.

[ebidel]

@mohanrohith I don' think the screenshot came through. Code looks great though!

[mohanrohith]

@ebidel updated with screenshot

[patrickhulce]

nice work @mohanrohith!

keep in mind that if you choose to use DevTools throttling over the techniques in advanced throttling guide, the latency values are only applied per request and not per packet/roundtrip. The latency values in your object look a little like RTT values rather than request latencies, so they likely won't match up to real-world performance (anecdotally we've found that applying roughly ~3x to the expected packet RTT to the overall request yields similar performance characteristics on average).