Lighthouse: Document do not run LH in parallel

[connorjclark]

Was provided this repo: https://github.com/y-a-v-a/lighthouse-async

run it async to see the error:

  lighthouse-async Error: no known mark: lh:gather:afterPass:OptimizedImages
  lighthouse-async     at Object.exports.stop (/Users/cjamcl/src/tmp/lighthouse-async/node_modules/marky/lib/marky.cjs.js:76:13)
  lighthouse-async     at Function.timeEnd (/Users/cjamcl/src/tmp/lighthouse-async/node_modules/lighthouse-logger/index.js:128:11)
  lighthouse-async     at Function.afterPass (/Users/cjamcl/src/tmp/lighthouse-async/node_modules/lighthouse/lighthouse-core/gather/gather-runner.js:407:11)
  lighthouse-async     at processTicksAndRejections (internal/process/task_queues.js:85:5)
  lighthouse-async     at async Function.runPass (/Users/cjamcl/src/tmp/lighthouse-async/node_modules/lighthouse/lighthouse-core/gather/gather-runner.js:665:5)
  lighthouse-async     at async Function.run (/Users/cjamcl/src/tmp/lighthouse-async/node_modules/lighthouse/lighthouse-core/gather/gather-runner.js:573:29)
  lighthouse-async     at async Function._gatherArtifactsFromBrowser (/Users/cjamcl/src/tmp/lighthouse-async/node_modules/lighthouse/lighthouse-core/runner.js:215:23)
  lighthouse-async     at async Function.run (/Users/cjamcl/src/tmp/lighthouse-async/node_modules/lighthouse/lighthouse-core/runner.js:81:21) +36ms

This is because the marks are stored in a global array, and gets cleared when a run finishes. If done while another run is between a start/end, the end will error.

I don't believe we'll fix the root cause here, since we do not support running Lighthouse like this. The suggested workaround is to run each Lighthouse run in a separate process. Even better, don't run Lighthouse in parallel with itself - the performance metrics will be affected.

[brendankenny]

singletons always come back to bite you (but yeah)

pending close?

[connorjclark]

I think we have enough to close-close :) We can point people to this issue if it comes up again in the wild.

[patrickhulce]

For the record, this method of running LH in parallel within the same process is also not supported due to a myriad of stability/variance/i18n/sentry reasons too, only some of which are also singleton-based :)

[y-a-v-a]

we do not support running Lighthouse like this

I think you should mention this on https://github.com/GoogleChrome/lighthouse/blob/master/docs/readme.md#using-programmatically

to run each Lighthouse run in a separate process

It might be nice to tell on that same documentation page how to do so.

Even better, don't run Lighthouse in parallel with itself - the performance metrics will be affected.

Also nice to mention on that documentaion page, IMHO. Or even at the top of the readme.md

[patrickhulce]

great suggestions @y-a-v-a ! anyone interested in a PR :)

[patrickhulce]

We consider this done by the programmatic examples and variability docs on minimum hardware recs.