Chromedp: Connecting to docker chromedp/headless-shell instance via RemoteAllocator

[gregtczap]

I would like to know how to connect to a chromedp/headless-shell docker instance as a remote allocator. I have seen a few comments recommending to run the go binary inside of the chromedp/headless-shell container as a solution, but it has also been implied this is not the only way to connect. Is it possible to have a go process connect to headless-shell through a running docker instance of chromedp/headless-shell but with the go binary running on the host instead of inside the headless-shell container?

The chromedp/headless-shell documentation gives this run command as an example: docker run -d -p 9222:9222 --rm --name headless-shell chromedp/headless-shell which I can only assume is given for the purpose of connecting to the instance from outside, is this correct?

It seems to me this was the intention of the RemoteAllocator interface. But the problem I am having is that I don't know what url string to pass to func NewRemoteAllocator(parent context.Context, url string). I have tried passing it ws://127.0.0.1:9222 but it errors with could not dial "ws://127.0.0.1:9222": unexpected HTTP response status: 404. I know the url should probably look something like ws://127.0.0.1:9222/devtools/browser/12939166-3db0-49cd-9aaa-56381ef91a7f but the last portion is dynamic and I do not know how to obtain that url from the docker instance running headless-shell.

[gregtczap]

There is also the following comment from a previous issue which leads me to believe what I am trying to do is possible. I can only assume the interface he was talking about was RemoteAllocator. If I am wrong about this please point me in the right direction.

Thanks for the reply. If it helps, the new API uses an interface to "allocate" browsers, and we'll add an allocator which starts Docker containers with specific versions of headless Chromium.

_Posted by @mvdan in https://github.com/chromedp/chromedp/issues/244#issuecomment-480583002_

[pflouiep]

This is exactly the issue I'm trying to resolve as well. My go process did took this exact approach before the ChromeDP api implementation was changed to used ChromeDP.ctx. Do you have a time frame for this new functionality?

[mvdan]

We haven't started implementing the executor which seamlessly uses Docker, but you should be able to use RemoteAllocator to connect to an existing chrome debugging protocol port/socket. If that doesn't work, please paste your code in this issue and I can try to help debug and fix it.

[vianhanif]

Hi @gregtczap... I'm trying to do the same currently in my project. following @mvdan last comment, do you still unable to make the connection to the remote allocator? would like to share your code snippet here if you successfully made it. I would really appreciate 🙏 Thanks

[wpajqz]

it's the example

https://gist.github.com/wpajqz/45789f7e99df782cca2cb9a926770a00

[maxkrasnov]

use :9222/json/version, /json/version return json with param webSocketDebuggerUrl - this is what you need

[ElAntagonista]

You could use something like the following:

func getDebugURL() string {
    resp, err := http.Get("http://localhost:9222/json/version")
    if err != nil {
        log.Fatal(err)
    }

    var result map[string]interface{}

    if err := json.NewDecoder(resp.Body).Decode(&result); err != nil {
        log.Fatal(err)
    }
    return result["webSocketDebuggerUrl"].(string)
}

[nagaran1]

Using puppeteer you can launch a chromium based browser and it will produce an output web socket url somethig like "ws://127.0.0.1:8080/devtools/browser/7a732fe0-594a-47d3-a8fa-7d48415954cd".

const puppeteer = require("puppeteer"); const chalk = require("chalk"); var fs = require("fs"); const error = chalk.bold.red; const success = chalk.keyword("green"); const port = 8080; (async () => { try { // open the headless browser var browser = await puppeteer.launch( { headless: true, devtools: true, args: ['--no-sandbox','--remote-debugging-port='+port], ignoreDefaultArgs: [ '--no-first-run'] }); console.log(success("Browser opened succesfully " + browser._connection.url())); } catch (err) { // Catch and display errors console.log(error(err)); await browser.close(); console.log(error("Browser Closed")); } })();

Later in golang code we can use it like this,

`flag.StringVar(&devToolWsUrl, "devtools-ws-url", "ws://127.0.0.1:8080/devtools/browser/7a732fe0-594a-47d3-a8fa-7d48415954cd", "DevTools Websocket URL")
flag.Parse()

actxt, cancelActxt := chromedp.NewRemoteAllocator(context.Background(), devToolWsUrl)
defer cancelActxt()

   ctxt, cancelCtxt := chromedp.NewContext(actxt) // create new tab
defer cancelCtxt()                             // close tab afterwards

if err := chromedp.Run(ctxt,
    chromedp.Navigate("https://example.com"),
    chromedp.Title(&title),
); err != nil {
    log.Fatalf("Failed: %v", err)
}`