Lighthouse-ci: Support custom "Listening" pattern for startServerCommand

[harmenjanssen]

Running lhci using a php local server does not seem to work: Lighthouse isn't picking up the server output.

With the following .lighthouserc.json:

{
  "ci": {
    "collect": {
      "numberOfRuns": 3,
      "startServerCommand": "php -S localhost:60000 ./server.php",
      "url": [
        "http://localhost:60000/nl/styleguide"
      ]
    }
  }
}

I get the following error:

$ lhci autorun
✅  .lighthouseci/ directory writable
✅  Configuration file found
✅  Chrome installation found
Healthcheck passed!

Started a web server with "php -S localhost:60000 ./server.php"...
WARNING: Timed out waiting for the server to start listening.
         Ensure the server prints "listening" when it is ready.

Even though the output from the server contains the word "listening":

$ php -S localhost:60000 ./server.php
PHP 7.3.7 Development Server started at Tue Dec  3 09:54:41 2019
Listening on http://localhost:60000
Document root is /Users/harmen/Sites/foo
Press Ctrl-C to quit.

Not sure who's at fault here, if I can debug this further please let me know, I'd like to help fix this.

[patrickhulce]

Just to be clear, the whole process doesn't work or just the timeout message happens erroneously? I'm not able to reproduce the server not working, but I do get the unexpected warning message.

[harmenjanssen]

You're right, the server is working while Lighthouse is running.
I do get a lot of warnings in the report, like the following snippet:

"runWarnings": [
    "Lighthouse was unable to reliably load the page you requested. Make sure you are testing the correct URL and that the server is properly responding to all requests. (Status code: 500)"
  ],
  "runtimeError": {
    "code": "ERRORED_DOCUMENT_REQUEST",
    "message": "Lighthouse was unable to reliably load the page you requested. Make sure you are testing the correct URL and that the server is properly responding to all requests. (Status code: 500)"
  }

But to be honest I'm very new to this and piecing it together, so that might as well be caused by something else.

[harmenjanssen]

Alright, I've created a reduced testcase and the runWarnings I was getting in my previous comment were my own fault.

So, to summarize, the issue is this: Lighthouse is erroneously timing out and showing an error, but is working fine otherwise.

[patrickhulce]

So after some investigation, it appears that the php command intentionally does not print that message when run in a non-interactive shell (such as in lhci use cases). You can reproduce this behavior yourself by trying out php -S localhost:5001 server.php | tee foo.txt and observing no output.

Since this is WAI from PHP's perspective, you'd need to find another way to start the PHP server that prints "Listening" to get the warning to disappear in LHCI. Hope this helps!

[harmenjanssen]

Really? That's interesting.
Actually I tried Lighthouse with Artisan as well (Laravel's PHP commandline tool), which doesn't print "Listening" ever (but does print something else), and that got me thinking: shouldn't there be a way to configure the string that Lighthouse will be looking for?

In any case: thanks for getting back on this! 👍

[patrickhulce]

shouldn't there be a way to configure the string that Lighthouse will be looking for?

Sure, that's a reasonable option! I'll reopen and convert to tracking that. Unfortunately, it wouldn't fix the php -S case since it prints nothing but a reasonable thing to have for other cases.

[harmenjanssen]

No, I know, that's going to need something else. I think artisan always prints _something_ however so in that case it's at least a solvable problem to a large user base.

Thanks!

[jzabala]

Hi @patrickhulce do you have a name for the new option?

I can lend you a hand with this one.

It looks like the only change would be here:

https://github.com/GoogleChrome/lighthouse-ci/blob/adfbd102aebaf954bbf67879f57e5d3fa9ee2eb6/packages/cli/src/collect/collect.js#L100

[patrickhulce]

Sure if you want to! :)

How about startServerReadyPattern? a string that gets new RegExp(options.startServerReadyPattern, 'i')'d and defaults to (listen|ready)

[jzabala]

Thanks @patrickhulce.

Working on this one 🙂