Lighthouse-ci: Support production monitoring use case, no git required

[patrickhulce]

Currently every build needs a .hash but we don't necessarily need one. We could switch to a .projectVersion and support timestamps instead?

[patrickhulce]

New better idea: report command just accepts a --hash-method flag of either git or timestamp which would do Date.now().toString(16) to generate a timestamp hash instead

[patrickhulce]

we're punting on non-git cases for MVP

[thasmo]

@patrickhulce Does that mean in general, that lighthouse-ci will work without a project's git repository when using a timestamp based versioning? It would be nice to just run lhci autorun with a config and send the results to the server; without the need for a local git repository at all. (e.g. it would make it possible to create regular snapshot reports for specific URLs without the need to run it in a git repo or ci environment)

[patrickhulce]

it would make it possible to create regular snapshot reports for specific URLs without the need to run it in a git repo or ci environment

This broader effort we refer to as "production monitoring" which is on our roadmap but requires many subtle but important changes to the types of UI and assumptions made in the CI server currently. We're discussing internally exactly how we want to handle this.

[AaronLayton]

Just placing my support behind this Issue

[Zizzamia]

@patrickhulce would this be just a new Project under the current UI?
Something like https://lhci-server.lovelhci.com/app/projects/my-app-prod?
One point I have been thinking, is also how we could adapt the UI for running different experiments, I will open a different issue for that conversation.

[patrickhulce]

@Zizzamia yeah you can basically do a new project and fake the build context to get this to work today.

Something like the below (copied from #225)

export LHCI_BUILD_CONTEXT__CURRENT_HASH=$(node -e "new Date().getTime().toString(16).split('').reverse().join('')")
export LHCI_BUILD_CONTEXT__COMMIT_TIME=$(date --iso-8601=seconds)
export LHCI_BUILD_CONTEXT__CURRENT_BRANCH=master
export LHCI_BUILD_CONTEXT__COMMIT_MESSAGE="Periodic run "
export LHCI_BUILD_CONTEXT__AUTHOR="No one <[email protected]>"
export LHCI_BUILD_CONTEXT__AVATAR_URL="https://picsum.photos/id/237/200/200"
lhci ...

[kunalnagar]

@patrickhulce I have a quick question on tracking production URLs. Before that, let me explain about the setup we have currently.

We have a repository with a lighthouserc.js file and a GitHub action that runs lhci autorun. The config file has the following structure:

module.exports = {
  ci: {
    collect: // localhost urls and server is started using `npm start`
    assert: ...
    upload: // to custom lhci server using build token, username and password
  },
}

After reading the documentation and this comment, I see we can add a server key. I'm curious about two things:

1. Where are these results uploaded to if you don't specify a storage key
2. How is the CRON set up exactly if multiple people keep pushing to the repo and the GitHub action keeps running with this config?

...or am I being completely off my rocker here 😅 Thanks!

[patrickhulce]

It seems you've mixed the two use cases together @kunalnagar.

342 was about monitoring production URLs without a git repository at all. You just configure server.psiCollectCron without any of the other collect/assert/upload pieces. It runs on your server not in the CI process.

Where are these results uploaded to if you don't specify a storage key

I'm not sure what you mean by storage key in the context of the config you shared. If you're talking about the server's configuration then you must specify a storage key or it will simply fail. There's no such thing as the server without any storage mechanism.

If you're talking about where does data go if you don't configure an upload step, then the answer is nowhere, it will just stay on in your local CI.

How is the CRON set up exactly if multiple people keep pushing to the repo and the GitHub action keeps running with this config?

Cron doesn't run from the CI process CLI, it runs on the server.

[kunalnagar]

@patrickhulce thanks a lot for the clarification!

1. If the psiCollectCron step runs on the server and it stores the results in a SQL database, can those results be viewed using the lighthouse UI like the ci steps? Or do we have to configure something custom to view them?

2. I think I found an issue with the export stuff you mentioned in this comment. This is only to do with the LHCI_BUILD_CONTEXT__CURRENT_HASH variable:

   • If you define the variables as mentioned in your comment, sometimes the CURRENT_HASH value is incorrect and it takes the .git hash.

   • I tried moving it to using the env: key in GitHub actions like so:

     env:
      LHCI_BUILD_CONTEXT__CURRENT_HASH: $(node -e "new Date().getTime().toString(16).split('').reverse().join('')")
     

     ...and it sends the command instead of the calculated string. Screenshot below:

     

Here's a simplified github action script that i am using as per your comment:

name: Check lighthouse (production build)

on:
  schedule:
    - cron: '0 */1 * * *' # every hour

jobs:
  lighthouseprd:
    runs-on: ubuntu-18.04
    steps:
      - uses: actions/checkout@v2
      - name: Export env variables
        run: |
          export LHCI_BUILD_CONTEXT__CURRENT_HASH=$(node -e "new Date().getTime().toString(16).split('').reverse().join('')")
          export LHCI_BUILD_CONTEXT__COMMIT_TIME=$(date --iso-8601=seconds)
          export LHCI_BUILD_CONTEXT__CURRENT_BRANCH=master
          export LHCI_BUILD_CONTEXT__COMMIT_MESSAGE="CRON Periodic run"
          export LHCI_BUILD_CONTEXT__AUTHOR="Kunal Nagar <[email protected]>"
          export LHCI_BUILD_CONTEXT__AVATAR_URL="image_link_here"
      - name: Run lighthouse
        run: |
          sudo npm install -g @lhci/[email protected]
          lhci autorun // more config here

[patrickhulce]

This has ventured a little too far into "build my setup" territory sorry @kunalnagar 😅

[kunalnagar]

@patrickhulce no worries thanks for all the help! 🙂

I was just trying to understand if there's an issue with how Github Actions handles environment variables OR how the LHCI_BUILD_CONTEXT__CURRENT_HASH is implemented on lighthouse-ci - because the time (which is also calculated via bash) seems to go out correctly to the server, it's just the hash that is being taken from git.