Lighthouse: Unpredictable behavior of Lighthouse 4.3.1/5.1.0 in docker container

[dkajtoch]

Provide the steps to reproduce

1. Run LH on

What is the current behavior?

I am trying to run lighthouse from inside Docker container and I am seeing strange unpredictable behavior when executing the following command in bash (after getting inside):
lighthouse --disable-storage-reset --output=json --no-enable-error-reporting --chrome-flags="--no-sandbox --headless --disable-gpu" https://www.agencjawhites.pl

I build two docker containers with different versions of nodejs and lighthouse. The first one has nodejs==10.16.0, google-chrome==75.0.3770.100, lighthouse==5.1.0

FROM ubuntu:bionic

# install node js
RUN apt-get update \
    && apt-get install -y \
    build-essential \
    make \ 
    curl \
    ca-certificates \
    apt-transport-https \
    lsb-release \
    gnupg

# install google-chrome
RUN curl -sL https://dl.google.com/linux/direct/google-chrome-stable_current_amd64.deb -o google-chrome-stable.deb \
    && apt-get install -y ./google-chrome-stable.deb 

# create user
# Add Chrome as a user
RUN groupadd -r chrome && useradd -r -g chrome chrome \
    && mkdir /home/chrome && chown -R chrome:chrome /home/chrome

RUN curl -sL https://deb.nodesource.com/setup_10.x | bash - \
    && apt-get install -y nodejs 

# install lighthouse
RUN npm install -g [email protected]

WORKDIR /home/chrome

# Run Chrome non-privileged
USER chrome

The second one has nodejs==8.16.0, google-chrome==75.0.3770.100, lighthouse==4.3.1

FROM ubuntu:bionic

# install node js
RUN apt-get update \
    && apt-get install -y \
    build-essential \
    make \ 
    curl \
    ca-certificates \
    apt-transport-https \
    lsb-release \
    gnupg

# install google-chrome
RUN curl -sL https://dl.google.com/linux/direct/google-chrome-stable_current_amd64.deb -o google-chrome-stable.deb \
    && apt-get install -y ./google-chrome-stable.deb 

# create user
# Add Chrome as a user
RUN groupadd -r chrome && useradd -r -g chrome chrome \
    && mkdir /home/chrome && chown -R chrome:chrome /home/chrome

RUN curl -sL https://deb.nodesource.com/setup_8.x | bash - \
    && apt-get install -y nodejs 

# install lighthouse
RUN npm install -g [email protected]

WORKDIR /home/chrome

# Run Chrome non-privileged
USER chrome

My docker version:
Docker version 18.09.7, build 2d0083d

I am running the same lighthouse command (stated earlier) in both containers against the https://www.agencjawhites.pl (which has redirect). Sometimes lighthouse==5.1.0 finishes without errors, but sometimes hangs (see screenshot) and produces errors. The same things happens with ligthouse==4.3.1. Sometimes it gives full report but sometimes it produces errors. I am not seeing any pattern so that is why it is unpredictable. I have no idea what might be causing this. I have ligthouse==4.3.1 installed on my local computer (linux mint tara, nodejs==8.12.0, google-chrome==75.0.3770.100) and each time I run lighthouse it finishes without errors, whereas in docker it is a lottery. I wanted to run lighthouse on Kubernetes Pod, but right now with such unpredictable behavior it is not an option. Any ideas what might be causing this? I collected a few screenshots with errors:

What is the expected behavior?

Environment Information

• Affected Channels:
• Lighthouse version:
• Node.js version:
• Operating System:

Related issues

[patrickhulce]

Nothing helpful really to add here other than it appears Chrome randomly hangs in Docker environments with no clear and reproducible pattern as to why. There's nothing really that can be done about it from the Lighthouse/client perspective AFAIK.

Related #9212.

[dkajtoch]

Thanks for quick answer @patrickhulce. So this is a problem with google chrome + docker. Do you know if moving to some docker alternatives like rkt may solve the problem? Where should I report this issue? Google Chrome? Docker?

[patrickhulce]

I'm not sure about rkt. We see a lot of scaled LH setups running actual VMs rather than containers for the most stable results.

If you could manage to find a minified repro with a clear and reproducible pattern, then crbug.com would be the place to file it :)

[skadeglad]

Might be not related, but to stop chrome from crashing in docker-selenium containers, it is generally required to use host's shared memory (either via mounting /dev/shm or via a flag --shm-size=2g )
see: https://github.com/SeleniumHQ/docker-selenium#running-the-images

[dkajtoch]

Thanks @patrickhulce and @skadeglad. I will put it on my list to try out.

[patrickhulce]

Super helpful info @skadeglad thank you!

[dkajtoch]

@patrickhulce I am not sure if you saw this project: LighthouseBot. They seemed to be able to run lighthouse inside Docker container. I haven't found any bug on their issue list so maybe it will work.

[patrickhulce]

I'm familiar with the project, and you certainly can run inside docker it just appears it has much higher error rates than running with more resources outside docker (like we're bumping into here).

[exterkamp]

We don't have the bandwidth to investigate this more, we would need some help bisecting this problem to track down where this broke.

If anyone comes up with a good solution, please post here with details! Thanks.

[dkajtoch]

I performed tests on more than 50 thousand different urls and lighthouse was responsive thanks to @skadeglad solution. Flag --shm-size=2g works for me.

[iAmServerless]

I think this should be added to the documentation. wasted around 2 weeks to fix this.

[connorjclark]

@ashuanindian can you send a PR?

[iAmServerless]

@connorjclark
Sorry, I lost your message in GitHub notifications.
Please find the PR https://github.com/GoogleChromeLabs/lighthousebot/pull/91/

[patrickhulce]

Thanks very much @ashuanindian!

I will say though that the lighthousebot project is deprecated and I'm not sure how many eyes it gets, if you wanted to have an even bigger impact you could open one to update https://github.com/GoogleChrome/lighthouse-ci/tree/master/docs/recipes/docker-client too :D

We can also update it if you'd prefer to just keep your contributions to lighthousebot though.