Pagedown: chrome_print() throws error "Error in get_entrypoint (debug_port): Cannot connect R to Chrome. Please retry."

[gershomtripp]

The specific issue is related to this SO question. You can also read my answer here.

The problem is that chrome_print is inconsistently connecting to Chrome. My solution is to putchrome_print in a repeat loop, in which case it always works, albeit sometimes only after multiple attempts. Ideally it would connect the first time, every time.

This may or may not be related, but I also keep getting this warning: [2019-12-20 12:36:33] [warning] got non-close frame while closing

[cderv]

Hi,

to completely understand what is going on here, can you please send us the log messages activating verbose argument ?

Here the to types:

• First, verbose = 1 to get first some messages from pagedown.
• Then another one using verbose = 2 to get also messages about wbesocket and chrome.

You'll be able to attach them here.

I am surprised the check about the protocol being reachable works but websocket connexion to the page does not. This check tries 20 times max by default to connect to chrome and in your case it seems to work because your error message about get_entrypoint comes after that.

Thanks!

[RLesur]

I am surprised the check about the protocol being reachable works but websocket connexion to the page does not.

Me too! I've never had this error.
I guess that when get_entrypoint() tries to obtain the websocket url, no target (tab) has been created yet by the browser.

IMO there are two workarounds:

• try to get the Page websocket URL _n_ times like in is_remote_protocol_ok().

• adopt the same workflow as in Puppeteer: connect to the browser websocket url, create and attach a target in flatten mode and send all the commands with the sessionId parameter (if I understand well the flatten mode but I've never tried it). There is much more work for this option.

[gershomtripp]

No problem.

---

Input → verbose = 1

pagedown::chrome_print("https://stackoverflow.com/questions/25166624/insert-picture-table-in-r-markdown", verbose = 1)

Output

Using the browser "/Applications/Google Chrome.app/Contents/MacOS/Google Chrome"
Checking the remote connection in 20 attempts.
Connected at attempt 2
Closing websocket connection
Closing browser
Cleaning browser working directory

---

Input → verbose = 2

pagedown::chrome_print("https://stackoverflow.com/questions/25166624/insert-picture-table-in-r-markdown", verbose = 2)

Output

Using the browser "/Applications/Google Chrome.app/Contents/MacOS/Google Chrome"
Checking the remote connection in 20 attempts.
Connected at attempt 2
Fehler in get_entrypoint(debug_port) : 
  Cannot connect R to Chrome. Please retry.
Closing browser
Cleaning browser working directory

[RLesur]

Hi @gershomtripp,

Happy new year!
Have you seen #159? Would it be possible for you to test this PR?

Thanks!

[gershomtripp]

159 fixes the issue for me. Thanks for the help @RLesur and @cderv !

One thing that hasn't changed is the warning I mention above. The script runs fine, but it still throws a lot of these warnings, even after the fix. It's also odd that the minutes change for each warning, but not the seconds. These are the warnings thrown while scraping 50 pages:

Let me know if I should open a new issue.

[RLesur]

These warnings mean that one (or many) websocket connection was not closed when headless Chrome is killed.

The order of execution of cleanup instructions is important. I tried to improve it in https://github.com/rstudio/pagedown/commit/329dc3d59b2461a44c0e5ed5d40375f456a0f819 but it seems this is not sufficient. I wonder whether this is related to the use of a private event loop (#127).

Let me know if I should open a new issue.

@gershomtripp Yes, please! It would be great if you could provide a minimal reproducible example because I haven't these warnings. Thanks!