Lighthouse: Problem running containerized Lighthouse node 8 app in OpenShift
Not _exactly_ related to Lighthouse, but I'm hoping someone can point me in the right direction.
A simple node 8 application using the lighthouse and chrome-launcher npm modules to generate reports against a given URL.
Everything works locally, as well as docker-ized. However, when the same docker image is deployed in _OpenShift_, it doesn't work.
Since everything runs with lowest permissions level in OpenShift, could this be some permissions issue? Or maybe some environment variable/parameter for Chrome is not set correctly, causing it to not start in headless mode etc?
Error Log
Lighthouse Test Service running on :::3000
Chrome Path: /usr/bin/google-chrome
Url to test: https://github.com
Fri, 15 Dec 2017 21:23:57 GMT ChromeLauncher:verbose created /tmp
Fri, 15 Dec 2017 21:23:57 GMT ChromeLauncher:verbose Launching with command:
"/usr/bin/google-chrome" --disable-translate --disable-background-networking --safebrowsing-disable-auto-update --disable-sync --metrics-recording-only --disable-default-apps --mute-audio --no-first-run --remote-debugging-port=36509 --user-data-dir=/tmp --disable-setuid-sandbox --headless --disable-gpu --no-sandbox about:blank
Fri, 15 Dec 2017 21:23:57 GMT ChromeLauncher:verbose Chrome running with pid 6156 on port 36509.
Fri, 15 Dec 2017 21:23:57 GMT ChromeLauncher Waiting for browser.
Fri, 15 Dec 2017 21:23:57 GMT ChromeLauncher Waiting for browser...
Fri, 15 Dec 2017 21:23:57 GMT ChromeLauncher Waiting for browser.....
Fri, 15 Dec 2017 21:23:58 GMT ChromeLauncher Waiting for browser.......
Fri, 15 Dec 2017 21:23:59 GMT ChromeLauncher Waiting for browser.........
Fri, 15 Dec 2017 21:24:00 GMT ChromeLauncher Waiting for browser...........
Fri, 15 Dec 2017 21:24:00 GMT ChromeLauncher Waiting for browser.............
Fri, 15 Dec 2017 21:24:01 GMT ChromeLauncher Waiting for browser...............
Fri, 15 Dec 2017 21:24:03 GMT ChromeLauncher Waiting for browser.................
Fri, 15 Dec 2017 21:24:04 GMT ChromeLauncher Waiting for browser...................
Fri, 15 Dec 2017 21:24:05 GMT ChromeLauncher Waiting for browser.....................
Fri, 15 Dec 2017 21:24:06 GMT ChromeLauncher Waiting for browser.....................✓
Chrome listening on port: 36509
CWD: /
Fri, 15 Dec 2017 21:25:38 GMT CriConnection:error Timeout waiting for initial Debugger Protocol connection.
Fri, 15 Dec 2017 21:25:43 GMT CriConnection:warn Cannot create new tab; reusing open tab.
Fri, 15 Dec 2017 21:26:01 GMT CriConnection:error Timeout waiting for initial Debugger Protocol connection.
Fri, 15 Dec 2017 21:26:05 GMT status Disconnecting from browser...
Fri, 15 Dec 2017 21:26:07 GMT CriConnection:warn disconnect() was called without an established connection.
{ Error: Timeout waiting for initial Debugger Protocol connection.
at ClientRequest.request.setTimeout._ (/node_modules/lighthouse/lighthouse-core/gather/connections/cri.js:118:21)
at Object.onceWrapper (events.js:313:30)
at emitNone (events.js:106:13)
at ClientRequest.emit (events.js:208:7)
at Socket.emitTimeout (_http_client.js:708:34)
at Object.onceWrapper (events.js:313:30)
at emitNone (events.js:106:13)
at Socket.emit (events.js:208:7)
at Socket._onTimeout (net.js:407:8)
at ontimeout (timers.js:475:11) code: 'CRI_TIMEOUT' }
These errors, specifically, look suspicious:
Fri, 15 Dec 2017 21:25:38 GMT CriConnection:error Timeout waiting for initial Debugger Protocol connection.
Fri, 15 Dec 2017 21:25:43 GMT CriConnection:warn Cannot create new tab; reusing open tab.
printenv (removed all the kubernetes/openshift vars)
I have no name!@lighthousedemo-1-pw9nq:/$ printenv
NODE_VERSION=8.9.3
HOSTNAME=lighthousedemo-1-pw9nq
TERM=xterm
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
PWD=/
SHLVL=1
APP_PORT=3000
HOME=/
YARN_VERSION=1.3.2
CI=true
I have no name!@lighthousedemo-1-pw9nq:/$ google-chrome --version
mkdir: cannot create directory '//.local': Permission denied
touch: cannot touch '//.local/share/applications/mimeapps.list': No such file or directory
Google Chrome 64.0.3282.24 dev
ps
I have no name!@lighthousedemo-1-pw9nq:/$ ps -ef
UID PID PPID C STIME TTY TIME CMD
1000690+ 1 0 0 20:09 ? 00:00:04 node index.js
1000690+ 30 1 0 20:09 ? 00:00:00 [cat] <defunct>
1000690+ 31 1 0 20:09 ? 00:00:00 [cat] <defunct>
1000690+ 5829 0 0 21:20 ? 00:00:00 /bin/sh -i -c TERM=xterm /bin/sh
1000690+ 5835 5829 0 21:20 ? 00:00:00 /bin/sh
1000690+ 5836 5835 0 21:20 ? 00:00:00 bash
1000690+ 6156 1 4 21:23 ? 00:00:05 /usr/bin/google-chrome --disable-translate --disable-background-networking
1000690+ 6166 6156 0 21:23 ? 00:00:00 cat
1000690+ 6167 6156 0 21:23 ? 00:00:00 cat
1000690+ 6169 6156 0 21:23 ? 00:00:00 /opt/google/chrome-unstable/chrome --type=zygote --headless --no-sandbox -
1000690+ 6219 6169 0 21:24 ? 00:00:00 /opt/google/chrome-unstable/chrome --type=renderer --no-sandbox --use-gl=s
1000690+ 6241 6169 0 21:25 ? 00:00:00 /opt/google/chrome-unstable/chrome --type=renderer --no-sandbox --use-gl=s
kodeninja
All 11 comments
Dockerfile used
FROM node:8-slim
# Install utilities
RUN apt-get update --fix-missing && apt-get -yq upgrade
# Install latest chrome dev package.
RUN wget -q -O - https://dl-ssl.google.com/linux/linux_signing_key.pub | apt-key add - \
&& sh -c 'echo "deb [arch=amd64] http://dl.google.com/linux/chrome/deb/ stable main" >> /etc/apt/sources.list.d/google.list' \
&& apt-get update \
&& apt-get install -yq google-chrome-unstable --no-install-recommends \
&& rm -rf /var/lib/apt/lists/* \
&& rm -rf /src/*.deb
ADD https://github.com/Yelp/dumb-init/releases/download/v1.2.0/dumb-init_1.2.0_amd64 /usr/local/bin/dumb-init
RUN chmod +x /usr/local/bin/dumb-init
# Download latest Lighthouse from npm.
# cache bust so we always get the latest version of LH when building the image.
ARG CACHEBUST=1
COPY package.json .
RUN npm i
# Add the simple server.
COPY . .
# Add a chrome user and setup home dir.
RUN groupadd -r chrome && useradd -r -m -g chrome -G audio,video chrome && \
mkdir -p /home/chrome/reports && chmod -R 777 /home/chrome && chmod -R 777 /tmp && \
chown -R chrome:chrome /home/chrome
USER chrome
#VOLUME /home/chrome/reports
#WORKDIR /home/chrome/reports
# Disable Lighthouse error reporting to prevent prompt.
ENV CI=true
ENTRYPOINT ["node", "index.js"]
Digging in further, when I try to launch chrome _manually_ with the same command that chrome-launcher is issuing, I see the following errors:
I have no name!@lighthousedemo-3-jxmfq:/$ "/usr/bin/google-chrome-stable" --disable-translate --disable-background-networking --safebrowsing-disable-auto-update --disable-sync --metrics-recording-only --disable-default-apps --mute-audio --no-first-run --remote-debugging-port=43995 --user-data-dir=/tmp --disable-setuid-sandbox --headless --disable-gpu --no-sandbox --remote-debugging-address=0.0.0.0 about:blank
mkdir: cannot create directory '//.local': Permission denied
touch: cannot touch '//.local/share/applications/mimeapps.list': No such file or directory
DevTools listening on ws://0.0.0.0:43995/devtools/browser/099b7c9b-cbcf-422c-a7cf-0b4cf4cf372d
[1215/224734.561286:ERROR:nss_util.cc(83)] Failed to create /.pki/nssdb directory.
So, definitely permissions errors. Not sure if these are what are causing lighthouse to barf.
kodeninja
on 16 Dec 2017
Nice job sleuthing so far!! Not sure exactly what's going on, but the error you're receiving seems like lighthouse never connected to Chrome even though chrome-launcher was able to.
It looks like Chrome takes a long time to boot up though. If your script looks like our using programmatically sample, could you try injecting some additional wait time between when chrome-launcher is able to connect (i.e. when launch() resolves) and you try to run lighthouse to see if that helps.
You can also keep Lighthouse out of the equation entirely in a script to debug. Something like the below? (note: not actually tested may need tweaking 😄 )
const http = require('http')
const chromeLauncher = require('chrome-launcher');
const SECONDS_TO_WAIT = 10 // try increasing this number til it works?
async function run() {
console.log('launching...')
const chrome = await chromeLauncher.launch()
console.log('giving chrome some time...')
await new Promise(resolve => setTimeout(resolve, SECONDS_TO_WAIT * 1000))
console.log('trying http request...')
http.get({hostname: 'localhost', port: chrome.port, path: '/json/new'}, resp => {
resp.on('data', console.log)
resp.on('end', () => console.log('ended!'))
})
}
run().catch(console.error)
patrickhulce
on 16 Dec 2017
Ok, I think I got around the permissions errors by leveraging k8s's emptyDir volume mount feature.
...
spec:
volumes:
- name: chrome-dir
emptyDir: {}
containers:
- name: node-lighthousedemo
image: 'lighthouse-reports:74063be0'
ports:
- name: app
containerPort: 3000
protocol: TCP
env:
- name: APP_PORT
value: '3000'
resources: {}
volumeMounts:
- name: chrome-dir
mountPath: /.local
- name: chrome-dir
mountPath: /.pki
...
Now, when I try to manually run chrome from the container, I don't see those errors anymore.
I have no name!@lighthousedemo-4-dhnsd:/tmp$ "/usr/bin/google-chrome-stable" --disable-translate --disable-background-
networking --safebrowsing-disable-auto-update --disable-sync --metrics-recording-only --disable-default-apps --mute-audio --
no-first-run --remote-debugging-port=36403 --user-data-dir=/tmp --disable-setuid-sandbox --headless --disable-gpu --no-sandb
ox --remote-debugging-address=0.0.0.0 about:blank
DevTools listening on ws://0.0.0.0:36403/devtools/browser/b2e0b872-a32c-46ed-853e-41a86c144730
@patrickhulce I just tried something similar. To the manually launched headless chrome instance above, I tried connecting from a local lighthouse cli (I'm using oc port-forward under the hood to connect to the container from my local):
$ oc port-forward lighthousedemo-4-dhnsd 36403:36403
Forwarding from 127.0.0.1:36403 -> 36403
Forwarding from [::1]:36403 -> 36403
Handling connection for 36403
Handling connection for 36403
Handling connection for 36403
Handling connection for 36403
Handling connection for 36403
Handling connection for 36403
Handling connection for 36403
Handling connection for 36403
Handling connection for 36403
$ lighthouse "https://github.com" --port=36403 --chrome-flags="--headless" --hostname="0.0.0.0"
CriConnection:error Timeout waiting for initial Debugger Protocol connection. +0ms
CriConnection:warn Cannot create new tab; reusing open tab. +1ms
CriConnection:error Timeout waiting for initial Debugger Protocol connection. +10s
status Disconnecting from browser... +0ms
CriConnection:warn disconnect() was called without an established connection. +1ms
Debugger protocol timed out while connecting to Chrome.
lighthouse still times out with the same error.
kodeninja
on 16 Dec 2017
Hmmmm, does running the script I posted without Lighthouse succeed under the same circumstances? Either with programmatic chrome launcher or without?
patrickhulce
on 16 Dec 2017
@patrickhulce I'm trying that now.
BTW, running the _exact_ same chrome launch command _locally_, and then connecting to it via the lighthouse cli works!
Local chrome headless
$ /Applications/Google\ Chrome.app/Contents/MacOS/Google\ Chrome --disable-translate --disable-background-networking --safebrowsing-disable-auto-update --disable-sync --metrics-recording-only --disable-default-apps --mute-audio --no-first-run --remote-debugging-port=23333 --user-data-dir=/tmp --disable-setuid-sandbox --headless --disable-gpu --no-sandbox --remote-debugging-address=0.0.0.0 about:blank
DevTools listening on ws://0.0.0.0:23333/devtools/browser/2667429c-5cd3-49a2-a810-c896fbd6a1ab
$ lighthouse "https://github.com" --port=23333 --chrome-flags="--headless" --hostname="0.0.0.0"
status Initializing… +0ms
status Loading page & waiting for onload URL, Viewport, ViewportDimensions, ThemeColor, Manifest, RuntimeExceptions, ChromeConsoleMessages, ImageUsage, Accessibility, EventListeners, AnchorsWithNoRelNoopener, AppCacheManifest, DOMStats, JSLibraries, OptimizedImages, PasswordInputsWithPreventedPaste, ResponseCompression, TagsBlockingFirstPaint, WebSQL, MetaDescription, FontSize, CrawlableLinks, MetaRobots, Hreflang +556ms
statusEnd Loading page & waiting for onload +20s
...
@patrickhulce used your script to connect to the remote headless chrome above (listening on WS port 36403):
test.js
"use strict";
const http = require('http');
//const chromeLauncher = require('chrome-launcher');
//const SECONDS_TO_WAIT = 10; // try increasing this number til it works?
async function run() {
// console.log('launching...');
// const chrome = await chromeLauncher.launch();
//
// console.log('giving chrome some time...');
// await new Promise(resolve => setTimeout(resolve, SECONDS_TO_WAIT * 1000));
console.log('trying http request...');
http.get({hostname: 'localhost', port: +process.argv[2], path: '/json/new'}, resp => {
resp.setEncoding('utf8');
resp.on('data', (data) => {
console.log("Got data!");
console.log("%s", data);
});
resp.on('end', () => console.log('ended!'));
})
}
run().catch(console.error);
$ node test.js 36403
trying http request...
Got data!
{
"description": "",
"devtoolsFrontendUrl": "/devtools/inspector.html?ws=localhost:36403/devtools/page/(BCC4CB96EA8F960F2132ED85FE0F1B3D)",
"id": "(BCC4CB96EA8F960F2132ED85FE0F1B3D)",
"title": "",
"type": "page",
"url": "about:blank",
"webSocketDebuggerUrl": "ws://localhost:36403/devtools/page/(BCC4CB96EA8F960F2132ED85FE0F1B3D)"
}
ended!
@patrickhulce so, from the above, it looks like that call succeeded?
kodeninja
on 18 Dec 2017
@patrickhulce nvm, I got it to work! I hadn't put in the resources section in the OpenShift deployment config, so there wasn't enough room for Chrome to come up and work 😁.
Thanks for all your help!
kodeninja
on 19 Dec 2017
👍1
Glad you got it sorted out and thanks for sharing the solution for future puzzled folks :)
patrickhulce
on 19 Dec 2017
🎉1
Was this page helpful?
0 / 5 - 0 ratings
Related issues
motiejuss
·
3Comments
asistapl
·
3Comments
johnfrancisli
·
3Comments
etelai
·
3Comments
workjalexanderfox
·
3Comments