Lighthouse-ci: Add support and docs for Gitlab CI
Trying to use lhci in Gitlab-CI. I am getting a CHROME_PATH must be set... error. See logs below:
291 $ yarn test:lighthouse
292 yarn run v1.19.1
293 $ lhci autorun
294 ✅ .lighthouseci/ directory writable
295 ✅ Configuration file found
296 ⚠️ GitHub token not set
297 Healthcheck passed!
298 Automatically determined ./public as `staticDistDir`.
299 Set it explicitly in lighthouserc.json if incorrect.
300 Started a web server on port 38319...
301 Running Lighthouse 3 time(s) on http://localhost:38319/404.html
302 Run #1...failed!
303 Error: Lighthouse failed with exit code 1
304 at ChildProcess.process.on.code (/builds/origenstudio/gatsby-starter-default/node_modules/@lhci/cli/src/collect/lighthouse-runner.js:102:21)
305 at ChildProcess.emit (events.js:198:13)
306 at Process.ChildProcess._handle.onexit (internal/child_process.js:248:12)
307 Runtime error encountered: The environment variable CHROME_PATH must be set to executable of a build of Chromium version 54.0 or later.
I've tried to use a service to install chrome. I have used the chromedp/headless-shell:latest docker image, however the error remains.
Has anyone configure lhci in Gitlab?
PolGuixe
All 20 comments
Could you give it a try with export CHROME_PATH=/headless-shell/headless-shell? FWIW, I'm not 100% sure the headless shell will support all of the Lighthouse commands since it looks like it's not pure headless Chrome but a modified/stripped down version, but it's worth a shot.
patrickhulce
on 18 Nov 2019
Also, you might need to set --collect.settings.chromeFlags="--no-sandbox".
If you get it working would you be willing to share your config so we can document Gitlab CI setups? :)
patrickhulce
on 18 Nov 2019
More than happy to share my setup. First I need to get it working 😉
PolGuixe
on 18 Nov 2019
@patrickhulce I haven't been lucky. Still the same error:
My current gitlab-ci.yml is:
image: node:10
variables:
CHROME_PATH: /headless-shell/headless-shell?FWIW
cache:
key: ${CI_COMMIT_REF_SLUG}
paths:
- node_modules/
- plugins/
- non_ES5_node_modules
stages:
- setup
- test
- build
- post-build-test
- deploy
setup:
stage: setup
script:
- yarn
only:
- master
- triggers
- merge_requests
- develop
test:
stage: test
script:
- yarn test
only:
- merge_requests
- develop
- master
build:
stage: build
script:
- yarn build
artifacts:
paths:
- public
- src/schema.flow.js
- schema.json
only:
- merge_requests
- develop
- master
- triggers
post-build-test:
stage: post-build-test
services:
- chromedp/headless-shell:latest
script:
- yarn test:report
- yarn test:lighthouse
dependencies:
- build
artifacts:
paths:
- .lighthouseci
- .eslint-report
reports:
junit: .eslint-report/eslint-junit.xml
only:
- merge_requests
- develop
- master
the scripts in package.json are:
"scripts": {
"install": "node ./src/scripts/install-submodules.js ./plugins",
"postinstall": "sh -c \"yarn --silent are-you-es5 check -r . | tail -n 1 > ./non_ES5_node_modules \"",
"build": "NODE_ICU_DATA=node_modules/full-icu gatsby build",
"develop": "gatsby develop",
"debug": "node --inspect node_modules/.bin/gatsby",
"start": "npm run develop",
"serve": "gatsby serve",
"netlify:preview": "netlify deploy",
"clean": "gatsby clean",
"format": "prettier --write \"src/**/*.js\"",
"flow": "flow",
"flow:coverage": "flow-coverage-report -i 'src/**/*.js' -i 'src/**/*.jsx' -x 'src/scripts/*' -x 'src/schema.flow.js' --threshold 75",
"eslint": "eslint src/ --ext js,jsx",
"eslint:report": "ESLINT_JUNIT_OUTPUT='.eslint-report/eslint-junit.xml' eslint src/ --ext js,jsx -f ./node_modules/eslint-junit/index.js",
"test": "yarn eslint && yarn flow && yarn flow:coverage && echo \"Write more tests! -> https://gatsby.app/unit-testing\"",
"test:report": "yarn eslint:report && yarn flow && yarn flow:coverage && echo \"Write more tests! -> https://gatsby.app/unit-testing\"",
"test:jest": "yarn jest",
"test:lighthouse": "lhci autorun --collect.settings.chromeFlags='--no-sandbox'",
"purge-production-cache": "node ./src/scripts/purge-production-cache"
},
Well you'll definitely want to get rid of the ?FWIW dangling on CHROME_PATH but looking through the rest of chrome-launcher code I'm not sure it's ever going to work with that docker image you have.
Instead, what if you gave the circleci/node:10.16-browsers docker image a try? I know you're not using circle but their docker images are just freely available on dockerhub and that's an image that's known to work already. You can get rid of the CHROME_PATH variable if you try this.
EDIT: it looks like you've been trying to set the docker image in the services: key of gitlab CI but from my brief reading of their docs that looks like it will only mount the image and expose ports, but not run the commands in that image which is what we need here. You'd likely need to change the root image at the top to run in the circleci/node:10.16-browsers to see any effect.
patrickhulce
on 18 Nov 2019
True... I was treating services like a addons in Travis.
I did some more testing with the following images:
circleci/node:10.16-browsers: does work in my case I don't get write permission so certain processes in my CI work -->cannot create ./non_ES5_node_modules: Permission deniedcypress/browsers:node10.16.0-chrome76: 🎊 it worked! 🎊
Do you think that using the cypress images is the best approach or should we create new ones?
PolGuixe
on 18 Nov 2019
Do you think that using the cypress images is the best approach or should we create new ones?
Anything that is likely to be well maintained seems good to me! Cypress meets that bar right now, so I say 👍
The only hesitation I have is that it looks like the Chrome version is baked into the tag name. On the one hand that's good news for pinning a Lighthouse version like the rest of the script will, on the other hand it'll run behind stable pretty quickly. IMO, it's good enough to include in our getting started docs. Interested in a PR for the minimal Gitlab CI config? :)
patrickhulce
on 18 Nov 2019
Should be good enough for now. If someone find a better solutions. Please share it.
PolGuixe
on 19 Nov 2019
I did a test with cypress/browsers:node10.16.0-chrome77 and didn't worked.
Error:
$ lhci autorun
287 ✅ .lighthouseci/ directory writable
288 ✅ Configuration file found
289 ⚠️ GitHub token not set
290 Healthcheck passed!
291 Automatically determined ./public as `staticDistDir`.
292 Set it explicitly in lighthouserc.json if incorrect.
293 Started a web server on port 33735...
294 Running Lighthouse 3 time(s) on http://localhost:33735/404.html
295 Run #1...failed!
296 Error: Lighthouse failed with exit code 1
297 at ChildProcess.process.on.code (/builds/origenstudio/gatsby-starter-default/node_modules/@lhci/cli/src/collect/lighthouse-runner.js:102:21)
298 at ChildProcess.emit (events.js:198:13)
299 at Process.ChildProcess._handle.onexit (internal/child_process.js:248:12)
300 Mon, 18 Nov 2019 23:36:38 GMT ChromeLauncher Waiting for browser.
301 Mon, 18 Nov 2019 23:36:38 GMT ChromeLauncher Waiting for browser...
302 Mon, 18 Nov 2019 23:36:39 GMT ChromeLauncher Waiting for browser.....
303 Mon, 18 Nov 2019 23:36:39 GMT ChromeLauncher Waiting for browser.......
304 Mon, 18 Nov 2019 23:36:40 GMT ChromeLauncher Waiting for browser.........
305 Mon, 18 Nov 2019 23:36:40 GMT ChromeLauncher Waiting for browser...........
306 Mon, 18 Nov 2019 23:36:41 GMT ChromeLauncher Waiting for browser.............
307 Mon, 18 Nov 2019 23:36:41 GMT ChromeLauncher Waiting for browser...............
...
Hm so Chrome never successfully launched then? You might still need the --collect.settings.chromeFlags="--no-sandbox" bit depending on the host OS.
You could always try installing the deps yourself too, we're about to add this to a "Jenkins setup" guide, basically if it's ubuntu-based...
```bash
Chrome apt-key
echo "deb [arch=amd64] http://dl.google.com/linux/chrome/deb/ stable main" | sudo tee -a /etc/apt/sources.list.d/google.list
wget -q -O - https://dl.google.com/linux/linux_signing_key.pub | sudo apt-key add -
Node apt-key
curl -sL https://deb.nodesource.com/setup_10.x | sudo -E bash -
Install dependencies
sudo apt-get update
sudo apt-get install -y nodejs google-chrome-stable
patrickhulce
on 19 Nov 2019
My fault. I removed --collect.settings.chromeFlags="--no-sandbox" in a previous commit and forgot to test. Then updated chrome version and send to test.
Using --collect.settings.chromeFlags="--no-sandbox" works with both versions cypress/browsers:node10.16.0-chrome77 and cypress/browsers:node10.16.0-chrome76.
Config files look like these - I have simplified them to avoid unnecessary noise -:
.gitlab-ci.yml
image: cypress/browsers:node10.16.0-chrome77
cache:
key: ${CI_COMMIT_REF_SLUG}
paths:
- node_modules/
stages:
- setup
- test
- build
- post-build-test
- deploy
setup:
stage: setup
script:
- yarn
only:
- merge_requests
test:
stage: test
script:
- yarn test
only:
- merge_requests
build:
stage: build
script:
- yarn build
artifacts:
paths:
- public
only:
- merge_requests
post-build-test:
stage: post-build-test
script:
- yarn test:report
- yarn test:lighthouse
dependencies:
- build
artifacts:
paths:
- .lighthouseci
- .eslint-report
reports:
junit: .eslint-report/eslint-junit.xml
only:
- merge_requests
packge.json
{
...
"scripts": {
"test:lighthouse": "lhci autorun --collect.settings.chromeFlags='--no-sandbox'",
...
},
...
}
@patrickhulce I am reopening. Let me know if I should create different issue.
While trying to upload the results to a LHCI server from Gitlab-CI I got the following error:
Error: Unable to determine current branch with `git rev-parse --abbrev-ref HEAD`
303 at getCurrentBranchRaw_ (/builds/origenstudio/gatsby-starter-default/node_modules/@lhci/utils/src/build-context.js:83:11)
304 at getCurrentBranch (/builds/origenstudio/gatsby-starter-default/node_modules/@lhci/utils/src/build-context.js:93:18)
305 at runLHCITarget (/builds/origenstudio/gatsby-starter-default/node_modules/@lhci/cli/src/upload/upload.js:222:18)
306 at process._tickCallback (internal/process/next_tick.js:68:7)
307 WARNING: upload command failed.
Here is a related issue: https://gitlab.com/gitlab-org/gitlab-foss/issues/19421
PolGuixe
on 19 Nov 2019
Thanks for that issue link @PolGuixe! This is true of a couple CI providers that work on a detached HEAD. For now the workaround is the same as that issue, manually running git checkout -B "$CI_BUILD_REF_NAME" "$CI_BUILD_REF" until we get support for gitlab's specific env vars into the CLI.
patrickhulce
on 19 Nov 2019
@patrickhulce I guess that this git command needs to be runned before the the lhci command like? 🤔
script:
- yarn test:report
- git checkout -B "$CI_BUILD_REF_NAME" "$CI_BUILD_REF"
- yarn test:lighthouse
Correct, the current git hash needs to have a branch label before it can be uploaded to a LHCI server.
patrickhulce
on 20 Nov 2019
I can confirm that it worked. 🎊
PolGuixe
on 21 Nov 2019
I'm having the same issue.
Solved by using circleci image and adding sudo to npm install -g @lhci/[email protected]
perf_test:
image: circleci/node:10.16-browsers
stage: test
script:
- npm install
- npm run build
- sudo npm install -g @lhci/[email protected]
- |
lhci collect \
--url=https://example.com \
--upload.target=temporary-public-storage \
--collect.settings.chromeFlags="--no-sandbox" \
|| echo "LHCI failed!"
Lusitaniae
on 11 Dec 2019
There's additional issues,
circleci images are running with unprivileged user which will cause issues
- npm install global
- running chrome (lighthouse)
Furthermore, their images start Xvfb in advance which seems to conflict with lighthouse when it tries to manage this itself.
Also some issues are difficult to replicate locally.
Amended working Gitlab CI config:
perf_test:
image: circleci/node:10.16-browsers
timeout: 10m
stage: test
script:
- npm install
- npm run build
- sudo su root
- killall Xvfb
- npm install -g @lhci/[email protected]
- |
lhci collect \
--url=https://example.com \
--upload.target=temporary-public-storage \
--collect.settings.chromeFlags="--no-sandbox" \
|| echo "LHCI failed!"
Earlier issues:
Running Lighthouse 3 time(s) on https://example.com
191 Run #1...done.
192 Run #2...done.
193 Run #3...done.
194 Done running Lighthouse!
196
(EE)
00:02
197 Fatal server error:
198 (EE) Server is already active for display 99
199 If this server is no longer running, remove /tmp/.X99-lock
200 and start again.
201 (EE)
In case it is helpful to someone, I have written about how I integrated lighthouse-ci and lighthouse ci server to my gitlab CI workflow as a proof of concept.
Here is the article: https://medium.com/@gaurav5430/web-performance-integrating-lightouse-ci-in-your-gitlab-ci-cd-pipeline-129c788591bd?source=friends_link&sk=ef3fdc7c3146ceae3432dd3b09c6e243
gaurav5430
on 6 Jun 2020
Related issues
WebCloud
·
4Comments
manoj-mass
·
3Comments
JackMidd-Amido
·
4Comments
klapec
·
3Comments
lgummadi7
·
3Comments
Most helpful comment
In case it is helpful to someone, I have written about how I integrated lighthouse-ci and lighthouse ci server to my gitlab CI workflow as a proof of concept.
Here is the article: https://medium.com/@gaurav5430/web-performance-integrating-lightouse-ci-in-your-gitlab-ci-cd-pipeline-129c788591bd?source=friends_link&sk=ef3fdc7c3146ceae3432dd3b09c6e243