Playwright: [Question] Run playwright from docker container running express server - sandbox issues

[rat-matheson]

Issue:
I am unable to configure sandbox usage in my environment.

• Is it possible to set the user to be used for launching the chrome browser? I'd like to run my server with a higher privileged user but the browser with a non-privileged user
• Perhaps there is a way to create a sandbox for my root user? However, I've heard that chrome will not launch this way (https://crbug.com/638180)
• Any suggestions as to how I can set this up given my environment below?

Env

• express server using playwright to parse external websites running in docker container
• I've used your dockerfile as the base for creating my image with some variations
  
  
  
  • other packages are installed including my server package
  
  
  • I start my server using pm2 as a launcher. The server itself is what uses playwright
  
  
  • I have disabled the non-privileged user step in your dockerfile because the server needs to do privileged things (eg, I've commented out USER pwuser). As a result, I believe my chrome instance cannot find a sandbox
  
  

Thanks for all the hard work and the great library!

[yury-s]

The best way to solve this would be to run as non-privileged user as otherwise bad things may happen. You should only run without sandbox if you fully trust the web pages you are loading. But I guess there is a good reason it cannot be done.

I assume you are disabling sandbox via command line arguments, in that case chromium should not complain about sandbox. Can you share custom arguments you are passing to the browser? Also can you launch with DEBUG=pw:browser* and share the log?

[rat-matheson]

Thanks for the response!

I have made some progress. I updated my server to work when run with a non-privileged user and I manually installed playwright using node_modules/playright/install.js after I built my docker image.
However, the browser is still unable to launch due to no sandbox being available

How I try to launch

const browser = await playwright.chromium.launch();
const context = await browser.newContext();
const page = await context.newPage();

error

2020-05-28T20:30:14.468Z pw:browser /home/pwuser/.cache/ms-playwright/chromium-764964/chrome-linux/chrome --disable-background-networking --enable-features=NetworkService,NetworkServiceInProcess --disable-background-timer-throttling --disable-backgrounding-occluded-windows --disable-breakpad --disable-client-side-phishing-detection --disable-component-extensions-with-background-pages --disable-default-apps --disable-dev-shm-usage --disable-extensions --disable-features=TranslateUI,BlinkGenPropertyTrees,ImprovedCookieControls,SameSiteByDefaultCookies --disable-hang-monitor --disable-ipc-flooding-protection --disable-popup-blocking --disable-prompt-on-repost --disable-renderer-backgrounding --disable-sync --force-color-profile=srgb --metrics-recording-only --no-first-run --enable-automation --password-store=basic --use-mock-keychain --user-data-dir=/tmp/playwright_dev_profile-wcxIAg --remote-debugging-pipe --headless --hide-scrollbars --mute-audio --no-startup-window
2020-05-28T20:30:14.480Z pw:browser pid=86
2020-05-28T20:30:15.186Z pw:browser:err [0528/203015.003458:FATAL:zygote_host_impl_linux.cc(116)] No usable sandbox! Update your kernel or see https://chromium.googlesource.com/chromium/src/+/master/docs/linux/suid_sandbox_development.md for more information on developing with the SUID sandbox. If you want to live dangerously and need an immediate workaround, you can try using --no-sandbox.
2020-05-28T20:30:15.186Z pw:browser:err #0 0x55fc5ba5a8a9 base::debug::CollectStackTrace()
2020-05-28T20:30:15.186Z pw:browser:err #1 0x55fc5b9bf7f3 base::debug::StackTrace::StackTrace()
2020-05-28T20:30:15.186Z pw:browser:err #2 0x55fc5b9d0280 logging::LogMessage::~LogMessage()
2020-05-28T20:30:15.186Z pw:browser:err #3 0x55fc5d22d86e service_manager::ZygoteHostImpl::Init()
2020-05-28T20:30:15.186Z pw:browser:err #4 0x55fc5b5a2faf content::ContentMainRunnerImpl::Initialize()
2020-05-28T20:30:15.186Z pw:browser:err #5 0x55fc5b603885 service_manager::Main()
2020-05-28T20:30:15.186Z pw:browser:err #6 0x55fc5b5a1571 content::ContentMain()
2020-05-28T20:30:15.186Z pw:browser:err #7 0x55fc5b60259d headless::(anonymous namespace)::RunContentMain()
2020-05-28T20:30:15.187Z pw:browser:err #8 0x55fc5b60229c headless::HeadlessShellMain()
2020-05-28T20:30:15.187Z pw:browser:err #9 0x55fc58fc1527 ChromeMain
2020-05-28T20:30:15.187Z pw:browser:err #10 0x7f23ce0c1b97 __libc_start_main
2020-05-28T20:30:15.187Z pw:browser:err #11 0x55fc58fc136a _start
2020-05-28T20:30:15.187Z pw:browser:err
2020-05-28T20:30:15.188Z pw:browser:err Received signal 6
2020-05-28T20:30:15.198Z pw:browser:err #0 0x55fc5ba5a8a9 base::debug::CollectStackTrace()
2020-05-28T20:30:15.207Z pw:browser:err #1 0x55fc5b9bf7f3 base::debug::StackTrace::StackTrace()
2020-05-28T20:30:15.220Z pw:browser:err #2 0x55fc5ba5a445 base::debug::(anonymous namespace)::StackDumpSignalHandler()
2020-05-28T20:30:15.221Z pw:browser:err #3 0x7f23d4756890 (/lib/x86_64-linux-gnu/libpthread-2.27.so+0x1288f)
2020-05-28T20:30:15.222Z pw:browser:err #4 0x7f23ce0dee97 gsignal
2020-05-28T20:30:15.223Z pw:browser:err #5 0x7f23ce0e0801 abort
2020-05-28T20:30:15.233Z pw:browser:err #6 0x55fc5ba593a5 base::debug::BreakDebugger()
2020-05-28T20:30:15.241Z pw:browser:err #7 0x55fc5b9d0720 logging::LogMessage::~LogMessage()
2020-05-28T20:30:15.254Z pw:browser:err #8 0x55fc5d22d86e service_manager::ZygoteHostImpl::Init()
2020-05-28T20:30:15.262Z pw:browser:err #9 0x55fc5b5a2faf content::ContentMainRunnerImpl::Initialize()
2020-05-28T20:30:15.270Z pw:browser:err #10 0x55fc5b603885 service_manager::Main()
2020-05-28T20:30:15.278Z pw:browser:err #11 0x55fc5b5a1571 content::ContentMain()
2020-05-28T20:30:15.286Z pw:browser:err #12 0x55fc5b60259d headless::(anonymous namespace)::RunContentMain()
2020-05-28T20:30:15.294Z pw:browser:err #13 0x55fc5b60229c headless::HeadlessShellMain()
2020-05-28T20:30:15.322Z pw:browser:err #14 0x55fc58fc1527 ChromeMain
2020-05-28T20:30:15.324Z pw:browser:err #15 0x7f23ce0c1b97 __libc_start_main
2020-05-28T20:30:15.347Z pw:browser:err #16 0x55fc58fc136a _start
2020-05-28T20:30:15.348Z pw:browser:err r8: 0000000000000000 r9: 00007ffdc4c6e120 r10: 0000000000000008 r11: 0000000000000246
2020-05-28T20:30:15.349Z pw:browser:err r12: 00007ffdc4c6f3f0 r13: 00007ffdc4c6e380 r14: 00007ffdc4c6f400 r15: aaaaaaaaaaaaaaaa
2020-05-28T20:30:15.350Z pw:browser:err di: 0000000000000002 si: 00007ffdc4c6e120 bp: 00007ffdc4c6e370 bx: 00007ffdc4c6ebb4
2020-05-28T20:30:15.351Z pw:browser:err dx: 0000000000000000 ax: 0000000000000000 cx: 00007f23ce0dee97 sp: 00007ffdc4c6e120
2020-05-28T20:30:15.352Z pw:browser:err ip: 00007f23ce0dee97 efl: 0000000000000246 cgf: 002b000000000033 erf: 0000000000000000
2020-05-28T20:30:15.353Z pw:browser:err trp: 0000000000000000 msk: 0000000000000000 cr2: 0000000000000000
2020-05-28T20:30:15.355Z pw:browser:err [end of stack trace]
2020-05-28T20:30:15.355Z pw:browser:err Calling _exit(1). Core file will not be generated.
Unable to start browser Error: Protocol error (Target.setAutoAttach): Target closed.
at /usr/app/node_modules/playwright-core/lib/chromium/crConnection.js:130:63
at new Promise ()
at CRSession.send (/usr/app/node_modules/playwright-core/lib/chromium/crConnection.js:129:16)
at CRSession. (/usr/app/node_modules/playwright-core/lib/helper.js:64:31)
at Function.connect (/usr/app/node_modules/playwright-core/lib/chromium/crBrowser.js:63:27)
at Chromium.launch (/usr/app/node_modules/playwright-core/lib/server/chromium.js:41:53)
Error from handler

So I'm trying to address the above issue using puppeteer...troubleshooting.md#setting-up-chrome-linux-sandbox. I believe the recommended approach (sudo sysctl -w kernel.unprivileged_userns_clone=1) doesn't work because I'm running in a container.

So that leaves the second method (Setup setuid sandbox). I followed the direction mostly with the following changes:

• used /home/pwuser/.cache/ms-playwright/chromium-764964/chrome-linux/ rather than /node_modules/puppeteer... because it didn't exist
• When copying the sandbox, I created it in a different location and named it chrome_pwuser_sandbox
• I made pwuser the owner of the sandbox

Here's the new error

2020-05-28T21:08:08.666Z pw:browser /home/pwuser/.cache/ms-playwright/chromium-764964/chrome-linux/chrome --disable-background-networking --enable-features=NetworkService,NetworkServiceInProcess --disable-background-timer-throttling --disable-backgrounding-occluded-windows --disable-breakpad --disable-client-side-phishing-detection --disable-component-extensions-with-background-pages --disable-default-apps --disable-dev-shm-usage --disable-extensions --disable-features=TranslateUI,BlinkGenPropertyTrees,ImprovedCookieControls,SameSiteByDefaultCookies --disable-hang-monitor --disable-ipc-flooding-protection --disable-popup-blocking --disable-prompt-on-repost --disable-renderer-backgrounding --disable-sync --force-color-profile=srgb --metrics-recording-only --no-first-run --enable-automation --password-store=basic --use-mock-keychain --user-data-dir=/tmp/playwright_dev_profile-HfQZVT --remote-debugging-pipe --headless --hide-scrollbars --mute-audio --no-startup-window
Unable to start browser Error: Failed to launch browser: Error: spawn /home/pwuser/.cache/ms-playwright/chromium-764964/chrome-linux/chrome ENOENT
at ChildProcess. (/usr/app/node_modules/playwright-core/lib/server/processLauncher.js:56:20)
at Object.onceWrapper (events.js:422:26)
at ChildProcess.emit (events.js:315:20)
at ChildProcess.EventEmitter.emit (domain.js:482:12)
at Process.ChildProcess._handle.onexit (internal/child_process.js:273:12)
at onErrorNT (internal/child_process.js:469:16)
at processTicksAndRejections (internal/process/task_queues.js:84:21)
Error from handler

Note that I also found that the The Linux SUID sandbox is almost but not completely removed from chromium/src.git/+/master/docs/linux/suid_sandbox_development.md. So perhaps this approach doesn't even make sense.

[boubkerbribri]

I think i have the same issue here with playwright

There is my docker file

# A minimal Docker image with Node and playwright
#
# Based upon:
# https://github.com/microsoft/playwright/blob/master/docs/docker/Dockerfile.bionic

FROM ubuntu:bionic

# 1. Install node12
RUN apt-get update && apt-get install -y curl && \
    curl -sL https://deb.nodesource.com/setup_12.x | bash - && \
    apt-get install -y nodejs

# 2. Install WebKit dependencies
RUN apt-get install -y libwoff1 \
                       libopus0 \
                       libwebp6 \
                       libwebpdemux2 \
                       libenchant1c2a \
                       libgudev-1.0-0 \
                       libsecret-1-0 \
                       libhyphen0 \
                       libgdk-pixbuf2.0-0 \
                       libegl1 \
                       libnotify4 \
                       libxslt1.1 \
                       libevent-2.1-6 \
                       libgles2 \
                       libvpx5

# 3. Install Chromium dependencies

RUN apt-get install -y libnss3 \
                       libxss1 \
                       libasound2

# 4. Install Firefox dependencies

RUN apt-get install -y libdbus-glib-1-2 \
                       libxt6


# 5. Copy some scripts 
...

# 6. Create folder and copy components
...

# 7. Install dependencies and force install playwright to download browsers
RUN npm install

npm install should install browsers but it does not

Error :

Unable to start browser Error: Failed to launch browser: Error: spawn /root/.cache/ms-playwright/chromium-764964/chrome-linux/chrome ENOENT

(same error with pwuser)

Runningnpm i [email protected] after npm install solved the problem for me

[rat-matheson]

Thanks for the further information. I believe that has helped me progress to the next step! However, I'm getting an error. First, though, here's my docker file

FROM ubuntu:bionic

WORKDIR /usr/app

RUN apt-get update && \
    apt-get install -y curl && \
    curl -sL https://deb.nodesource.com/setup_12.x | bash - && \
    apt-get install -y nodejs && \
    npm install -g pm2

# the following is required for running playwright which is used to scrape URLs
# 2. Install WebKit dependencies
RUN apt-get install -y libwoff1 \
    libopus0 \
    libwebp6 \
    libwebpdemux2 \
    libenchant1c2a \
    libgudev-1.0-0 \
    libsecret-1-0 \
    libhyphen0 \
    libgdk-pixbuf2.0-0 \
    libegl1 \
    libnotify4 \
    libxslt1.1 \
    libevent-2.1-6 \
    libgles2 \
    libvpx5

# 3. Install Chromium dependencies

RUN apt-get install -y libnss3 \
    libxss1 \
    libasound2

# 6. Add user so we don't need --no-sandbox in Chromium
RUN groupadd -r pwuser && useradd -r -g pwuser -G audio,video pwuser \
    && mkdir -p /home/pwuser/Downloads \
    && chown -R pwuser:pwuser /home/pwuser

# copy app specific files
...

# install app specific npm packages

# update workspace permissions so that server can run with pwuser
RUN chown -R pwuser:pwuser /usr/app

# Install playwright as pwuser.  This makes sure playright is installed to /home/pwuser/.cache.  If I don't do this, it installs to /root/.cache...the issue with this is that because I run as pwuser, playwright tries to launch from  /home/pwuser/.cache/ms-playwright/... and it is not found there if installed as root

USER pwuser
RUN npm i --only=prod [email protected] && \
    node node_modules/playwright/install.js

# Set up the sandbox
USER root
RUN cp /home/pwuser/.cache/ms-playwright/chromium-764964/chrome-linux/chrome_sandbox /home/chrome_sandbox && \
    chown root:root /home/chrome_sandbox && \
    chmod 4755 /home/chrome_sandbox
USER pwuser

# start my server
CMD ["pm2-runtime", "start", "process.json"]

When I launch my docker image, I set the environmental variable CHROME_DEVEL_SANDBOX

CHROME_DEVEL_SANDBOX=/home/chrome_sandbox

So, the most recent error I'm getting is

2020-06-02T14:52:11.588Z pw:browser /home/pwuser/.cache/ms-playwright/chromium-764964/chrome-linux/chrome --disable-background-networking --enable-features=NetworkService,NetworkServiceInProcess --disable-background-timer-throttling --disable-backgrounding-occluded-windows --disable-breakpad --disable-client-side-phishing-detection --disable-component-extensions-with-background-pages --disable-default-apps --disable-dev-shm-usage --disable-extensions --disable-features=TranslateUI,BlinkGenPropertyTrees,ImprovedCookieControls,SameSiteByDefaultCookies --disable-hang-monitor --disable-ipc-flooding-protection --disable-popup-blocking --disable-prompt-on-repost --disable-renderer-backgrounding --disable-sync --force-color-profile=srgb --metrics-recording-only --no-first-run --enable-automation --password-store=basic --use-mock-keychain --user-data-dir=/tmp/playwright_dev_profile-cLxHS4 --remote-debugging-pipe --headless --hide-scrollbars --mute-audio --no-startup-window
2020-06-02T14:52:11.676Z pw:browser pid=58
2020-06-02T14:52:12.136Z pw:browser:err [0602/145211.841401:FATAL:setuid_sandbox_host.cc(148)] The SUID sandbox helper binary is missing: /home/pwuser/chrome_sandbox Aborting now. See https://chromium.googlesource.com/chromium/src/+/master/docs/linux/suid_sandbox_development.md.
2020-06-02T14:52:12.136Z pw:browser:err #0 0x555e549e48a9 base::debug::CollectStackTrace()
2020-06-02T14:52:12.136Z pw:browser:err #1 0x555e549497f3 base::debug::StackTrace::StackTrace()
2020-06-02T14:52:12.136Z pw:browser:err #2 0x555e5495a280 logging::LogMessage::~LogMessage()
2020-06-02T14:52:12.136Z pw:browser:err #3 0x555e561b8c34 sandbox::SetuidSandboxHost::PrependWrapper()
2020-06-02T14:52:12.136Z pw:browser:err #4 0x555e561b7dbb service_manager::ZygoteHostImpl::LaunchZygote()
2020-06-02T14:52:12.136Z pw:browser:err #5 0x555e5452d8c8 content::(anonymous namespace)::LaunchZygoteHelper()
2020-06-02T14:52:12.136Z pw:browser:err #6 0x555e561b7026 service_manager::ZygoteCommunication::Init()
2020-06-02T14:52:12.137Z pw:browser:err #7 0x555e561bdca4 service_manager::CreateGenericZygote()
2020-06-02T14:52:12.137Z pw:browser:err #8 0x555e5452d03f content::ContentMainRunnerImpl::Initialize()
2020-06-02T14:52:12.137Z pw:browser:err #9 0x555e5458d885 service_manager::Main()
2020-06-02T14:52:12.137Z pw:browser:err #10 0x555e5452b571 content::ContentMain()
2020-06-02T14:52:12.137Z pw:browser:err #11 0x555e5458c59d headless::(anonymous namespace)::RunContentMain()
2020-06-02T14:52:12.137Z pw:browser:err #12 0x555e5458c29c headless::HeadlessShellMain()
2020-06-02T14:52:12.137Z pw:browser:err #13 0x555e51f4b527 ChromeMain
2020-06-02T14:52:12.137Z pw:browser:err #14 0x7fd1b7daeb97 __libc_start_main
2020-06-02T14:52:12.137Z pw:browser:err #15 0x555e51f4b36a _start
2020-06-02T14:52:12.137Z pw:browser:err
2020-06-02T14:52:12.140Z pw:browser:err Received signal 6
2020-06-02T14:52:12.164Z pw:browser:err #0 0x555e549e48a9 base::debug::CollectStackTrace()
2020-06-02T14:52:12.183Z pw:browser:err #1 0x555e549497f3 base::debug::StackTrace::StackTrace()
2020-06-02T14:52:12.206Z pw:browser:err #2 0x555e549e4445 base::debug::(anonymous namespace)::StackDumpSignalHandler()
2020-06-02T14:52:12.210Z pw:browser:err #3 0x7fd1be443890 (/lib/x86_64-linux-gnu/libpthread-2.27.so+0x1288f)
2020-06-02T14:52:12.212Z pw:browser:err #4 0x7fd1b7dcbe97 gsignal
2020-06-02T14:52:12.217Z pw:browser:err #5 0x7fd1b7dcd801 abort
2020-06-02T14:52:12.241Z pw:browser:err #6 0x555e549e33a5 base::debug::BreakDebugger()
2020-06-02T14:52:12.260Z pw:browser:err #7 0x555e5495a720 logging::LogMessage::~LogMessage()
2020-06-02T14:52:12.299Z pw:browser:err #8 0x555e561b8c34 sandbox::SetuidSandboxHost::PrependWrapper()
2020-06-02T14:52:12.357Z pw:browser:err #9 0x555e561b7dbb service_manager::ZygoteHostImpl::LaunchZygote()
2020-06-02T14:52:12.389Z pw:browser:err #10 0x555e5452d8c8 content::(anonymous namespace)::LaunchZygoteHelper()
2020-06-02T14:52:12.433Z pw:browser:err #11 0x555e561b7026 service_manager::ZygoteCommunication::Init()
2020-06-02T14:52:12.467Z pw:browser:err #12 0x555e561bdca4 service_manager::CreateGenericZygote()
2020-06-02T14:52:12.483Z pw:browser:err #13 0x555e5452d03f content::ContentMainRunnerImpl::Initialize()
2020-06-02T14:52:12.506Z pw:browser:err #14 0x555e5458d885 service_manager::Main()
2020-06-02T14:52:12.522Z pw:browser:err #15 0x555e5452b571 content::ContentMain()
2020-06-02T14:52:12.552Z pw:browser:err #16 0x555e5458c59d headless::(anonymous namespace)::RunContentMain()
2020-06-02T14:52:12.583Z pw:browser:err #17 0x555e5458c29c headless::HeadlessShellMain()
2020-06-02T14:52:12.635Z pw:browser:err #18 0x555e51f4b527 ChromeMain
2020-06-02T14:52:12.639Z pw:browser:err #19 0x7fd1b7daeb97 __libc_start_main
2020-06-02T14:52:12.687Z pw:browser:err #20 0x555e51f4b36a _start
2020-06-02T14:52:12.690Z pw:browser:err r8: 0000000000000000 r9: 00007ffcbb77b960 r10: 0000000000000008 r11: 0000000000000246
2020-06-02T14:52:12.693Z pw:browser:err r12: 00007ffcbb77ccd0 r13: 00007ffcbb77bbc0 r14: 00007ffcbb77cce0 r15: aaaaaaaaaaaaaaaa
2020-06-02T14:52:12.702Z pw:browser:err di: 0000000000000002 si: 00007ffcbb77b960 bp: 00007ffcbb77bbb0 bx: 00007ffcbb77c3f4
2020-06-02T14:52:12.703Z pw:browser:err dx: 0000000000000000 ax: 0000000000000000 cx: 00007fd1b7dcbe97 sp: 00007ffcbb77b960
2020-06-02T14:52:12.703Z pw:browser:err ip: 00007fd1b7dcbe97 efl: 0000000000000246 cgf: 002b000000000033 erf: 0000000000000000
2020-06-02T14:52:12.703Z pw:browser:err trp: 0000000000000000 msk: 0000000000000000 cr2: 0000000000000000
2020-06-02T14:52:12.703Z pw:browser:err [end of stack trace]
2020-06-02T14:52:12.703Z pw:browser:err Calling _exit(1). Core file will not be generated.
Unable to start browser Error: Protocol error (Target.setAutoAttach): Target closed.
at /usr/app/node_modules/playwright-core/lib/chromium/crConnection.js:130:63
at new Promise ()
at CRSession.send (/usr/app/node_modules/playwright-core/lib/chromium/crConnection.js:129:16)
at CRSession. (/usr/app/node_modules/playwright-core/lib/helper.js:64:31)
at Function.connect (/usr/app/node_modules/playwright-core/lib/chromium/crBrowser.js:63:27)
at Chromium.launch (/usr/app/node_modules/playwright-core/lib/server/chromium.js:41:53)

[rat-matheson]

Ok, I have a workaround. I switched to firefox and everything worked :)

[yury-s]

How I try to launch

const browser = await playwright.chromium.launch();

If you want to run this under root, you have to disable Chromium sanbox by passing corresponding arguments as the error message suggests: "If you want to live dangerously and need an immediate workaround, you can try using --no-sandbox."

I see that you switch to non-privileged user pwuser after installing all packages. In that case you shouldn't need any chrome_sandbox shenanigans . Does pm2-runtime still try to run playwright as root?

[aslushnikov]

I am unable to configure sandbox usage in my environment.

@rat-matheson we've updated instructions to launch Chromium in docker with sandbox. The TL;DR: in your case seem to be launching with a proper seccomp profile that you should download and use like this:

docker container run -it --rm --ipc=host --security-opt seccomp=seccomp_profile.json mcr.microsoft.com/playwright:bionic /bin/bash

As a last resort, in our upcoming v1.3 release, there's a new high-level launch option chromiumSandbox. You can easily run chromium without sandbox as a quick workaround:

// Launch chromium without Chromium Sandbox.
const browser = await chromium.launch({ chromiumSandbox: false });

Hope this helps!