Lighthouse-ci: Not seeing status checks despite passing LHCI_GITHUB_APP_TOKEN env

[klapec]

Hi,

I'm trying to run lighthouse-ci as part of my Github Actions workflow. I've got a following step defined:

- name: Run Lighthouse CI
  run: |
    npm install -g @lhci/[email protected]
    sh ./scripts/lighthouse-ci.sh
  env:
    LHCI_GITHUB_APP_TOKEN: ${{ secrets.LHCI_GITHUB_APP_TOKEN }}

and here's the lighthouse-ci script:

#!/bin/bash

yarn serve-ssr & npx wait-on http://localhost:3000/ping && lhci autorun \
  --upload.target=temporary-public-storage \
  --collect.url="http://localhost:3000/" \
  --collect.numberOfRuns=3

EXIT_CODE=$?
exit $EXIT_CODE

The job finishes successfully, and yet there are no status checks from lhci app on the pull request pages.
The Lighthouse CI GitHub app is installed.
Here's the lhci output:

2020-01-07T11:08:08.1615866Z ##[group]Run npm install -g @lhci/[email protected]
2020-01-07T11:08:08.1616571Z npm install -g @lhci/[email protected]
2020-01-07T11:08:08.1616905Z sh ./scripts/lighthouse-ci.sh
2020-01-07T11:08:08.1650457Z shell: /bin/bash -e {0}
2020-01-07T11:08:08.1651111Z env:
2020-01-07T11:08:08.1651413Z   SSH_AUTH_SOCK: /tmp/ssh-auth.sock
2020-01-07T11:08:08.1654217Z   LHCI_GITHUB_APP_TOKEN: ***
2020-01-07T11:08:08.1654571Z ##[endgroup]
2020-01-07T11:08:20.6725461Z /opt/hostedtoolcache/node/12.13.0/x64/bin/lhci -> /opt/hostedtoolcache/node/12.13.0/x64/lib/node_modules/@lhci/cli/src/cli.js
2020-01-07T11:08:20.7986075Z + @lhci/[email protected]
2020-01-07T11:08:20.7987248Z added 372 packages from 265 contributors in 12.174s
2020-01-07T11:08:21.2479204Z yarn run v1.19.2
2020-01-07T11:08:21.2922368Z $ node ./server/bootstrap.js
2020-01-07T11:08:28.6648453Z [11:08:28] WARN: build-info.properties not found
2020-01-07T11:08:38.8674730Z [11:08:38] INFO: Express server started on port 3000
2020-01-07T11:08:41.1089893Z ✅  .lighthouseci/ directory writable
2020-01-07T11:08:41.1093001Z ⚠️   Configuration file not found
2020-01-07T11:08:41.1818533Z ✅  Chrome installation found
2020-01-07T11:08:41.1824902Z Healthcheck passed!
2020-01-07T11:08:41.1890913Z 
2020-01-07T11:08:41.5886559Z Running Lighthouse 3 time(s) on http://localhost:3000/
2020-01-07T11:08:55.6706748Z Warning: componentWillMount has been renamed, and is not recommended for use. See https://fb.me/react-unsafe-component-lifecycles for details.
2020-01-07T11:08:55.6707696Z 
2020-01-07T11:08:55.6708622Z * Move code from componentWillMount to componentDidMount (preferred in most cases) or the constructor.
2020-01-07T11:08:55.6709336Z 
2020-01-07T11:08:55.6710135Z Please update the following components: LoadableComponent
2020-01-07T11:09:07.5682845Z Run #1...done.
2020-01-07T11:09:21.2047274Z Run #2...done.
2020-01-07T11:09:34.4405503Z Run #3...done.
2020-01-07T11:10:57.7562084Z Done running Lighthouse!
2020-01-07T11:10:57.7643052Z 
2020-01-07T11:10:59.1253930Z Uploading median LHR of http://localhost:3000/...success!
2020-01-07T11:10:59.1255873Z Open the report at https://storage.googleapis.com/lighthouse-infrastructure.appspot.com/reports/xxx.report.html
2020-01-07T11:11:00.8771080Z GitHub token found, attempting to set status...
2020-01-07T11:11:01.8443153Z GitHub accepted "success" status for "lhci/url/".
2020-01-07T11:11:03.3061416Z 
2020-01-07T11:11:03.3061848Z Done running autorun.

What am I doing wrong here?

[patrickhulce]

Hm, I'm not sure what's going on. GitHub accepted "success" status for "lhci/url/" means that the GitHub API responded with a 201 Created status code to our status check request, so at the least a status check is being created on some hash.

My best guess is that the hash that's being built in GitHub Actions is not the hash that the PR is for. If you add git rev-list --no-merges -n1 HEAD to your script, what hash does that output and does it match the hash of your commit in the PR?

[klapec]

@patrickhulce You're right, the hashes don't match. The script logs 589..., and the latest commit hash is actually 9d9....
I've added

with:
          ref: ${{ github.event.pull_request.head.sha }}

to the checkout action and now it works correctly.
Thanks, been scratching my head because of this for way too long. :)

[Yankovsky]

Thanks, it worked perfectly! To help further developers the fix goes like that:

- uses: actions/checkout@v2
  with:
    ref: ${{ github.event.pull_request.head.sha }}