Pagedown: When chrome_print() a page, what if the page is not accessible?

[yihui]

e.g. How can we deal with 404? Should chrome_print() signal an error?

[RLesur]

I had a quick look. I think this will be easy to solve.

If you inspect the log below, you can see that the response to the Page.navigate command (id: 4) is this json string:

{"id":4,"result":{"frameId":"EDBF02AD32BE8BCBE79A731216B79C98","loaderId":"EC7BB17B35D75139DED17E48AB67210C","errorText":"net::ERR_NAME_NOT_RESOLVED"}}

So we can test against msg$result$errorText in the print_pdf() function.
reference: https://chromedevtools.github.io/devtools-protocol/tot/Page#method-navigate

If we would want to obtain the http status code, I think we should enable the Network domain and track the loader. This would be much more complex for the same result.

_sorry, I'm in a hurry, I cannot do it now (but this will be trivial for you :smile:)_

pagedown::chrome_print('http://fake.web.site', wait=0, verbose = T)
#> Message received from headless Chrome: {"method":"Runtime.executionContextCreated","params":{"context":{"id":1,"origin":"://","name":"","auxData":{"isDefault":true,"type":"default","frameId":"EDBF02AD32BE8BCBE79A731216B79C98"}}}}
#> Message received from headless Chrome: {"id":1,"result":{}}
#> Message received from headless Chrome: {"id":2,"result":{}}
#> Message received from headless Chrome: {"id":3,"result":{}}
#> Message received from headless Chrome: {"id":4,"result":{"frameId":"EDBF02AD32BE8BCBE79A731216B79C98","loaderId":"EC7BB17B35D75139DED17E48AB67210C","errorText":"net::ERR_NAME_NOT_RESOLVED"}}
#> Message received from headless Chrome: {"method":"Runtime.executionContextsCleared","params":{}}
#> Message received from headless Chrome: {"method":"Page.frameStartedLoading","params":{"frameId":"EDBF02AD32BE8BCBE79A731216B79C98"}}
#> Message received from headless Chrome: {"method":"Runtime.executionContextsCleared","params":{}}
#> Message received from headless Chrome: {"method":"Page.frameNavigated","params":{"frame":{"id":"EDBF02AD32BE8BCBE79A731216B79C98","loaderId":"AD7F5FC4ABC912A2D2F4137F8E93E8B2","url":"chrome-error://chromewebdata/","securityOrigin":"://","mimeType":"text/html","unreachableUrl":"http://fake.web.site/"}}}
#> Message received from headless Chrome: {"method":"Runtime.executionContextCreated","params":{"context":{"id":1,"origin":"://","name":"","auxData":{"isDefault":true,"type":"default","frameId":"EDBF02AD32BE8BCBE79A731216B79C98"}}}}
#> Message received from headless Chrome: {"method":"Page.loadEventFired","params":{"timestamp":4069.283174}}
#> Message received from headless Chrome: {"method":"Page.frameStoppedLoading","params":{"frameId":"EDBF02AD32BE8BCBE79A731216B79C98"}}
#> Message received from headless Chrome: {"method":"Page.domContentEventFired","params":{"timestamp":4069.283539}}
#> Message received from headless Chrome: {"id":5,"result":{"result":{"type":"boolean","value":false}}}
#> Message received from headless Chrome: {"method":"Runtime.bindingCalled","params":{"name":"pagedownListener","payload":"","executionContextId":1}}
#> Message received from headless Chrome: {"id":6,"result":{"result":{"type":"object","subtype":"promise","className":"Promise","description":"Promise","objectId":"{\"injectedScriptId\":1,\"id\":1}"}}}
#> Message received from headless Chrome: {"method":"Page.frameResized","params":{}}
#> Message received from headless Chrome: {"method":"Page.frameResized","params":{}}
#> Message received from headless Chrome: {"id":7,"result":{"data":"JVBERi0xLjQKJdPr6eEKMSAwIG9iago8PC9DcmVhdG9yIChDaHJvbWl1bSkKL1Byb2R1Y2VyIChTa2lhL1BERiBtNzEpCi9DcmVhdGlvbkRhdGUgKEQ6MjAxOTAyMDgxMzM5NDArMDAnMDAnKQovTW9kRGF0ZSAoRDoyMDE5MDIwODEzMzk0MCswMCcwMCcpPj4KZW5kb2JqCjIgMCBvYmoKPDwvRmlsdGVyIC9GbGF0ZURlY29kZQovTGVuZ3RoIDExNj4+IHN0cmVhbQp4nG2NOw7CQBBD+zmFaySG+ax2dk+QOjQcAEEqkEjuL2U3KZICv8YubLN53QRpXPkUoxqeH/qRaubclXCy5iZc3B0uubBHTZhf9Ljg2zrOalEk8rZ7pP9b7UbRuQ/YzTzRbRBMC/V+JEMt2vffNDZWM8Yk5AplbmRzdHJlYW0KZW5kb2JqCjMgMCBvYmoKPDwvVHlwZSAvQ2F0YWxvZwovUGFnZXMgNCAwIFI+PgplbmRvYmoKNCAwIG9iago8PC9UeXBlIC9QYWdlcwovQ291bnQgMQovS2lkcyBbNSAwIFJdCi9SZXNvdXJjZXMgPDwvUHJvY1NldHMgWy9QREYgL1RleHQgL0ltYWdlQiAvSW1hZ2VDIC9JbWFnZUldPj4+PgplbmRvYmoKNSAwIG9iago8PC9UeXBlIC9QYWdlCi9SZXNvdXJjZXMgPDwvRXh0R1N0YXRlIDw8L0cwIDYgMCBSPj4+PgovTWVkaWFCb3ggWzAgMCA2MTIgNzkyXQovQ29udGVudHMgMiAwIFIKL1N0cnVjdFBhcmVudHMgMAovUGFyZW50IDQgMCBSPj4KZW5kb2JqCjYgMCBvYmoKPDwvY2EgMQovQk0gL05vcm1hbD4+CmVuZG9iagp4cmVmCjAgNwowMDAwMDAwMDAwIDY1NTM1IGYgCjAwMDAwMDAwMTUgMDAwMDAgbiAKMDAwMDAwMDE1NCAwMDAwMCBuIAowMDAwMDAwMzQwIDAwMDAwIG4gCjAwMDAwMDAzODcgMDAwMDAgbiAKMDAwMDAwMDUwNCAwMDAwMCBuIAowMDAwMDAwNjQ2IDAwMDAwIG4gCnRyYWlsZXIKPDwvU2l6ZSA3Ci9Sb290IDMgMCBSCi9JbmZvIDEgMCBSPj4Kc3RhcnR4cmVmCjY4MwolJUVPRg=="}}
#> Message received from headless Chrome: {"method":"Page.frameResized","params":{}}
#> Message received from headless Chrome: {"method":"Page.frameResized","params":{}}

^{Created on 2019-02-08 by the reprex package (v0.2.1.9000)}

[yihui]

No worries. I guess I can figure it out :) Thanks!

[yihui]

Errors like net::ERR_NAME_NOT_RESOLVED can be captured (05d9b2b), but 404 is not actually treated an error, e.g.

> pagedown::chrome_print('https://pagedown.rbind.io/html-letter2', verbose = T)

Message received from headless Chrome: {"method":"Runtime.executionContextCreated","params":{"context":{"id":1,"origin":"://","name":"","auxData":{"isDefault":true,"type":"default","frameId":"FB601E69C205E154AE3BB7266768FE2F"}}}}
Message received from headless Chrome: {"id":1,"result":{}}
Message received from headless Chrome: {"id":2,"result":{}}
Message received from headless Chrome: {"id":3,"result":{}}
Message received from headless Chrome: {"id":4,"result":{"frameId":"FB601E69C205E154AE3BB7266768FE2F","loaderId":"64958A829EB7B49D7BC2BD90BD6F6F1E"}}
Message received from headless Chrome: {"method":"Page.frameStartedLoading","params":{"frameId":"FB601E69C205E154AE3BB7266768FE2F"}}
Message received from headless Chrome: {"method":"Runtime.executionContextDestroyed","params":{"executionContextId":1}}
Message received from headless Chrome: {"method":"Runtime.executionContextsCleared","params":{}}
Message received from headless Chrome: {"method":"Page.frameNavigated","params":{"frame":{"id":"FB601E69C205E154AE3BB7266768FE2F","loaderId":"64958A829EB7B49D7BC2BD90BD6F6F1E","url":"https://pagedown.rbind.io/html-letter2","securityOrigin":"https://pagedown.rbind.io","mimeType":"text/html"}}}
Message received from headless Chrome: {"method":"Runtime.executionContextCreated","params":{"context":{"id":2,"origin":"https://pagedown.rbind.io","name":"","auxData":{"isDefault":true,"type":"default","frameId":"FB601E69C205E154AE3BB7266768FE2F"}}}}
Message received from headless Chrome: {"method":"Page.domContentEventFired","params":{"timestamp":64090.159762}}
Message received from headless Chrome: {"method":"Page.loadEventFired","params":{"timestamp":64090.159901}}
Message received from headless Chrome: {"method":"Page.frameStoppedLoading","params":{"frameId":"FB601E69C205E154AE3BB7266768FE2F"}}
Message received from headless Chrome: {"id":5,"result":{"result":{"type":"boolean","value":false}}}
Message received from headless Chrome: {"method":"Runtime.bindingCalled","params":{"name":"pagedownListener","payload":"","executionContextId":2}}
Message received from headless Chrome: {"id":6,"result":{"result":{"type":"object","subtype":"promise","className":"Promise","description":"Promise","objectId":"{\"injectedScriptId\":2,\"id\":1}"}}}
Message received from headless Chrome: {"method":"Page.frameResized","params":{}}
Message received from headless Chrome: {"method":"Page.frameResized","params":{}}
Message received from headless Chrome: {"id":7,"result":{"data":"JVBERi0xLjQKJdPr6eEKMSAwIG9iago8PC9DcmVhdG....
Message received from headless Chrome: {"method":"Page.frameResized","params":{}}
Message received from headless Chrome: {"method":"Page.frameResized","params":{}}

[RLesur]

oh! I forgot that with a 404 error, the webserver sends the 404 page. So, we need to deal with the Network domain I think.

[cderv]

Yes I think it needs to listen to Network.responseReceived event and checks for status in the message received.
I tested with crrri and its event emitter mechanism. I can see the 404 code. I'll try to add it in chrome_print.