This bug has been found via Webpagetest. I also did some tests with the command line.
The following possibilities instruct Chrome to ignore certificate errors and resolve www.lemonde.fr to IP 62.210.77.17. Note that the server 62.210.77.17 doesn't have a valid certificate for www.lemonde.fr.
Three possibilities :
lighthouse "https://www.lemonde.fr/?frz-demo=true" --chrome-flags="--ignore-certificate-errors --host-resolver-rules='MAP www.lemonde.fr 62.210.77.17'"
chrome --host-resolver-rules="MAP www.lemonde.fr 62.210.77.17" --remote-debugging-port=9251 --ignore-certificate-errors about:blank
lighthouse "https://www.lemonde.fr/?frz-demo=true" --port=9251
setDns www.lemonde.fr 62.210.77.17
navigate https://www.lemonde.fr/?frz-demo=true
Ref : http://wpt.fasterize.com/result/190221_X6_R/
The report is not launched whereas the page is correctly displayed on Chrome
Insecure document request: The URL you have provided does not have valid security credentials. This site is missing a valid, trusted certificate (net::ERR_CERT_COMMON_NAME_INVALID).
Full log :
status:verbose Create config +0ms
status:verbose Requiring gatherers +3ms
statusEnd:verbose Requiring gatherers +13ms
status:verbose Requiring audits +1ms
statusEnd:verbose Requiring audits +100ms
statusEnd:verbose Create config +9ms
status:verbose Runner setup +2ms
status Connecting to browser +1ms
statusEnd:verbose Connecting to browser +61ms
status:verbose Getting browser version +1ms
method => browser:verbose Browser.getVersion +3ms
method <= browser OK:verbose Browser.getVersion {"protocolVersion":"1.3","product":"Chrome/72.0.3626.109","revision":"@fae8db7ab9280fa6704a59980263c804f809ebd5","userAgent":"Mozilla/5.0 (Ma +9ms
statusEnd:verbose Getting browser version +1ms
status Resetting state with about:blank +0ms
Driver:verbose Network.enable +1ms
method => browser:verbose Network.enable +0ms
method <= browser OK:verbose Network.enable {} +7ms
Driver:verbose Page.enable +0ms
method => browser:verbose Page.enable +0ms
method <= browser OK:verbose Page.enable {} +1ms
method => browser:verbose Page.setLifecycleEventsEnabled {"enabled":true} +1ms
<= event:verbose Page.lifecycleEvent {"frameId":"01E01B67E93DDA6A4FB4F159E8BEE17D","loaderId":"2D64B8CDC534CD79B7D1B6FE3E282DA3","name":"commit","timestamp":103107.132125} +1ms
<= event:verbose Page.lifecycleEvent {"frameId":"01E01B67E93DDA6A4FB4F159E8BEE17D","loaderId":"2D64B8CDC534CD79B7D1B6FE3E282DA3","name":"DOMContentLoaded","timestamp":103107.134343} +0ms
<= event:verbose Page.lifecycleEvent {"frameId":"01E01B67E93DDA6A4FB4F159E8BEE17D","loaderId":"2D64B8CDC534CD79B7D1B6FE3E282DA3","name":"load","timestamp":103107.134789} +0ms
<= event:verbose Page.lifecycleEvent {"frameId":"01E01B67E93DDA6A4FB4F159E8BEE17D","loaderId":"2D64B8CDC534CD79B7D1B6FE3E282DA3","name":"networkAlmostIdle","timestamp":103107.135415} +1ms
<= event:verbose Page.lifecycleEvent {"frameId":"01E01B67E93DDA6A4FB4F159E8BEE17D","loaderId":"2D64B8CDC534CD79B7D1B6FE3E282DA3","name":"networkIdle","timestamp":103107.135415} +0ms
method <= browser OK:verbose Page.setLifecycleEventsEnabled {} +0ms
method => browser:verbose Emulation.setScriptExecutionDisabled {"value":false} +0ms
method <= browser OK:verbose Emulation.setScriptExecutionDisabled {} +0ms
method => browser:verbose Page.navigate {"url":"about:blank"} +1ms
listen once for event =>:verbose Page.frameNavigated +0ms
method <= browser OK:verbose Page.navigate {"frameId":"01E01B67E93DDA6A4FB4F159E8BEE17D","loaderId":"B96699B1C889E9AD77E9E6B3D5B37D24"} +6ms
<= event:verbose Page.frameStartedLoading {"frameId":"01E01B67E93DDA6A4FB4F159E8BEE17D"} +13ms
<= event:verbose Page.lifecycleEvent {"frameId":"01E01B67E93DDA6A4FB4F159E8BEE17D","loaderId":"B96699B1C889E9AD77E9E6B3D5B37D24","name":"init","timestamp":103107.188475} +0ms
<= event:verbose Page.frameNavigated {"frame":{"id":"01E01B67E93DDA6A4FB4F159E8BEE17D","loaderId":"B96699B1C889E9AD77E9E6B3D5B37D24","url":"about:blank","securityOrigin":"://","mimeType":"t +0ms
<= event:verbose Page.loadEventFired {"timestamp":103107.192467} +0ms
<= event:verbose Page.lifecycleEvent {"frameId":"01E01B67E93DDA6A4FB4F159E8BEE17D","loaderId":"B96699B1C889E9AD77E9E6B3D5B37D24","name":"load","timestamp":103107.192467} +0ms
<= event:verbose Page.frameStoppedLoading {"frameId":"01E01B67E93DDA6A4FB4F159E8BEE17D"} +0ms
<= event:verbose Page.domContentEventFired {"timestamp":103107.19318} +0ms
<= event:verbose Page.lifecycleEvent {"frameId":"01E01B67E93DDA6A4FB4F159E8BEE17D","loaderId":"B96699B1C889E9AD77E9E6B3D5B37D24","name":"DOMContentLoaded","timestamp":103107.19318} +1ms
statusEnd:verbose Resetting state with about:blank +0ms
status Benchmarking machine +0ms
method => browser:verbose Runtime.evaluate {"expression":"(function wrapInNativePromise() {\n const __nativePromise = window.__nativePromise || Promise;\n const URL = window._ +1ms
method <= browser OK:verbose Runtime.evaluate {"result":{"type":"number","value":1226,"description":"1226"}} +513ms
statusEnd:verbose Benchmarking machine +1ms
status Initializing… +0ms
listen once for event =>:verbose ServiceWorker.workerRegistrationUpdated +0ms
Driver:verbose ServiceWorker.enable +0ms
method => browser:verbose ServiceWorker.enable +0ms
method <= browser OK:verbose ServiceWorker.enable {} +1ms
<= event:verbose ServiceWorker.workerRegistrationUpdated {"registrations":[{"registrationId":"14","scopeURL":"https://www.youtube.com/","isDeleted":false},{"registrationId":"174","scopeURL" +1ms
Driver:verbose ServiceWorker.disable +1ms
method => browser:verbose ServiceWorker.disable +0ms
<= event:verbose ServiceWorker.workerVersionUpdated {"versions":[{"versionId":"4168","registrationId":"174","scriptURL":"https://www.reddit.com/sw.js","runningStatus":"stopped","status":"ac +0ms
method <= browser OK:verbose ServiceWorker.disable {} +0ms
listen for event =>:verbose ServiceWorker.workerVersionUpdated +0ms
Driver:verbose ServiceWorker.enable +0ms
method => browser:verbose ServiceWorker.enable +0ms
method <= browser OK:verbose ServiceWorker.enable {} +1ms
<= event:verbose ServiceWorker.workerRegistrationUpdated {"registrations":[{"registrationId":"14","scopeURL":"https://www.youtube.com/","isDeleted":false},{"registrationId":"174","scopeURL" +0ms
<= event:verbose ServiceWorker.workerVersionUpdated {"versions":[{"versionId":"4168","registrationId":"174","scriptURL":"https://www.reddit.com/sw.js","runningStatus":"stopped","status":"ac +0ms
Driver:verbose ServiceWorker.disable +1ms
method => browser:verbose ServiceWorker.disable +0ms
method <= browser OK:verbose ServiceWorker.disable {} +0ms
method => browser:verbose Emulation.setDeviceMetricsOverride {"mobile":true,"screenWidth":412,"screenHeight":660,"width":412,"height":660,"positionX":0,"positionY":0,"scale":1,"deviceScaleF +1ms
method => browser:verbose Network.setUserAgentOverride {"userAgent":"Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5 Build/MRA58N) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/71.0.3559.0 Mobil +0ms
method => browser:verbose Emulation.setTouchEmulationEnabled {"enabled":true} +0ms
<= event:verbose Page.frameResized {} +1ms
<= event:verbose Page.frameResized {} +0ms
method <= browser OK:verbose Emulation.setDeviceMetricsOverride {} +0ms
method <= browser OK:verbose Network.setUserAgentOverride {} +1ms
method <= browser OK:verbose Emulation.setTouchEmulationEnabled {} +0ms
method => browser:verbose Network.emulateNetworkConditions {"latency":0,"downloadThroughput":0,"uploadThroughput":0,"offline":false} +0ms
method <= browser OK:verbose Network.emulateNetworkConditions {} +1ms
Driver:verbose Runtime.enable +0ms
method => browser:verbose Runtime.enable +0ms
<= event:verbose Runtime.executionContextCreated {"context":{"id":1,"origin":"://","name":"","auxData":{"isDefault":true,"type":"default","frameId":"01E01B67E93DDA6A4FB4F159E8BEE17D"}}} +1ms
method <= browser OK:verbose Runtime.enable {} +0ms
method => browser:verbose Page.addScriptToEvaluateOnLoad {"scriptSource":"\n window.__nativePromise = Promise;\n window.__nativeError = Error;\n window.__nativeURL = UR +0ms
method <= browser OK:verbose Page.addScriptToEvaluateOnLoad {"identifier":"1"} +1ms
method => browser:verbose Page.addScriptToEvaluateOnLoad {"scriptSource":"(function registerPerformanceObserverInPage() {\n window.____lastLongTask = window.__perfNow();\n const observer +0ms
method <= browser OK:verbose Page.addScriptToEvaluateOnLoad {"identifier":"2"} +1ms
listen for event =>:verbose Page.javascriptDialogOpening +0ms
method => browser:verbose Storage.clearDataForOrigin {"origin":"https://www.lemonde.fr","storageTypes":"appcache,file_systems,indexeddb,local_storage,shader_cache,websql,service_workers,cac +0ms
method <= browser OK:verbose Storage.clearDataForOrigin {} +4ms
statusEnd:verbose Initializing… +0ms
method => browser:verbose Network.emulateNetworkConditions {"latency":0,"downloadThroughput":0,"uploadThroughput":0,"offline":false} +1ms
method <= browser OK:verbose Network.emulateNetworkConditions {} +0ms
status:verbose Running beforePass methods +0ms
method => browser:verbose Network.setBlockedURLs {"urls":[]} +0ms
method <= browser OK:verbose Network.setBlockedURLs {} +1ms
method => browser:verbose Network.setExtraHTTPHeaders {"headers":{"Cookie":"frz-demo=true"}} +0ms
method <= browser OK:verbose Network.setExtraHTTPHeaders {} +1ms
status:verbose Retrieving setup: Scripts +0ms
statusEnd:verbose Retrieving setup: Scripts +0ms
status:verbose Retrieving setup: CSSUsage +0ms
statusEnd:verbose Retrieving setup: CSSUsage +0ms
status:verbose Retrieving setup: ViewportDimensions +0ms
statusEnd:verbose Retrieving setup: ViewportDimensions +0ms
status:verbose Retrieving setup: Manifest +0ms
statusEnd:verbose Retrieving setup: Manifest +0ms
status:verbose Retrieving setup: RuntimeExceptions +0ms
listen for event =>:verbose Runtime.exceptionThrown +0ms
statusEnd:verbose Retrieving setup: RuntimeExceptions +0ms
status:verbose Retrieving setup: ChromeConsoleMessages +0ms
listen for event =>:verbose Log.entryAdded +0ms
Driver:verbose Log.enable +0ms
method => browser:verbose Log.enable +0ms
method <= browser OK:verbose Log.enable {} +1ms
method => browser:verbose Log.startViolationsReport {"config":[{"name":"discouragedAPIUse","threshold":-1}]} +1ms
method <= browser OK:verbose Log.startViolationsReport {} +0ms
statusEnd:verbose Retrieving setup: ChromeConsoleMessages +0ms
status:verbose Retrieving setup: Accessibility +0ms
statusEnd:verbose Retrieving setup: Accessibility +1ms
status:verbose Retrieving setup: ImageElements +0ms
statusEnd:verbose Retrieving setup: ImageElements +0ms
status:verbose Retrieving setup: LinkElements +0ms
statusEnd:verbose Retrieving setup: LinkElements +0ms
status:verbose Retrieving setup: MetaElements +0ms
statusEnd:verbose Retrieving setup: MetaElements +0ms
status:verbose Retrieving setup: AnchorsWithNoRelNoopener +0ms
statusEnd:verbose Retrieving setup: AnchorsWithNoRelNoopener +0ms
status:verbose Retrieving setup: AppCacheManifest +1ms
statusEnd:verbose Retrieving setup: AppCacheManifest +0ms
status:verbose Retrieving setup: Doctype +0ms
statusEnd:verbose Retrieving setup: Doctype +0ms
status:verbose Retrieving setup: DOMStats +0ms
statusEnd:verbose Retrieving setup: DOMStats +0ms
status:verbose Retrieving setup: JSLibraries +0ms
statusEnd:verbose Retrieving setup: JSLibraries +0ms
status:verbose Retrieving setup: OptimizedImages +1ms
statusEnd:verbose Retrieving setup: OptimizedImages +0ms
status:verbose Retrieving setup: PasswordInputsWithPreventedPaste +0ms
statusEnd:verbose Retrieving setup: PasswordInputsWithPreventedPaste +0ms
status:verbose Retrieving setup: ResponseCompression +0ms
statusEnd:verbose Retrieving setup: ResponseCompression +0ms
status:verbose Retrieving setup: TagsBlockingFirstPaint +0ms
method => browser:verbose Page.addScriptToEvaluateOnLoad {"scriptSource":"(function installMediaListener() {\n window.___linkMediaChanges = [];\n Object.defineProperty(HTMLLinkElement.pro +0ms
method <= browser OK:verbose Page.addScriptToEvaluateOnLoad {"identifier":"3"} +2ms
statusEnd:verbose Retrieving setup: TagsBlockingFirstPaint +0ms
status:verbose Retrieving setup: FontSize +0ms
statusEnd:verbose Retrieving setup: FontSize +0ms
status:verbose Retrieving setup: CrawlableLinks +0ms
statusEnd:verbose Retrieving setup: CrawlableLinks +0ms
status:verbose Retrieving setup: Hreflang +0ms
statusEnd:verbose Retrieving setup: Hreflang +0ms
status:verbose Retrieving setup: EmbeddedContent +0ms
statusEnd:verbose Retrieving setup: EmbeddedContent +1ms
status:verbose Retrieving setup: Canonical +0ms
statusEnd:verbose Retrieving setup: Canonical +0ms
status:verbose Retrieving setup: RobotsTxt +0ms
statusEnd:verbose Retrieving setup: RobotsTxt +0ms
statusEnd:verbose Running beforePass methods +0ms
status Loading page & waiting for onload Scripts, CSSUsage, ViewportDimensions, Manifest, RuntimeExceptions, ChromeConsoleMessages, Accessibility, ImageElements, LinkElements, MetaElements, AnchorsWithNoRelNoopener, AppCacheManifest, Doctype, DOMStats, JSLibraries, OptimizedImages, PasswordInputsWithPreventedPaste, ResponseCompression, TagsBlockingFirstPaint, FontSize, CrawlableLinks, Hreflang, EmbeddedContent, Canonical, RobotsTxt +0ms
method => browser:verbose Network.clearBrowserCache +0ms
method <= browser OK:verbose Network.clearBrowserCache {} +23ms
method => browser:verbose Network.setCacheDisabled {"cacheDisabled":true} +0ms
method <= browser OK:verbose Network.setCacheDisabled {} +1ms
method => browser:verbose Network.setCacheDisabled {"cacheDisabled":false} +1ms
method <= browser OK:verbose Network.setCacheDisabled {} +0ms
status:verbose Getting browser version +1ms
method => browser:verbose Browser.getVersion +0ms
method <= browser OK:verbose Browser.getVersion {"protocolVersion":"1.3","product":"Chrome/72.0.3626.109","revision":"@fae8db7ab9280fa6704a59980263c804f809ebd5","userAgent":"Mozilla/5.0 (Ma +0ms
statusEnd:verbose Getting browser version +0ms
method => browser:verbose Tracing.start {"categories":"-*,disabled-by-default-lighthouse,v8,v8.execute,blink.user_timing,blink.console,devtools.timeline,disabled-by-default-devtools.timelin +1ms
method <= browser OK:verbose Tracing.start {} +1ms
method => browser:verbose Page.setLifecycleEventsEnabled {"enabled":true} +1ms
<= event:verbose Page.lifecycleEvent {"frameId":"01E01B67E93DDA6A4FB4F159E8BEE17D","loaderId":"B96699B1C889E9AD77E9E6B3D5B37D24","name":"commit","timestamp":103107.188678} +0ms
<= event:verbose Page.lifecycleEvent {"frameId":"01E01B67E93DDA6A4FB4F159E8BEE17D","loaderId":"B96699B1C889E9AD77E9E6B3D5B37D24","name":"DOMContentLoaded","timestamp":103107.191912} +0ms
<= event:verbose Page.lifecycleEvent {"frameId":"01E01B67E93DDA6A4FB4F159E8BEE17D","loaderId":"B96699B1C889E9AD77E9E6B3D5B37D24","name":"load","timestamp":103107.192461} +0ms
<= event:verbose Page.lifecycleEvent {"frameId":"01E01B67E93DDA6A4FB4F159E8BEE17D","loaderId":"B96699B1C889E9AD77E9E6B3D5B37D24","name":"networkAlmostIdle","timestamp":103107.193209} +1ms
<= event:verbose Page.lifecycleEvent {"frameId":"01E01B67E93DDA6A4FB4F159E8BEE17D","loaderId":"B96699B1C889E9AD77E9E6B3D5B37D24","name":"networkIdle","timestamp":103107.193209} +0ms
method <= browser OK:verbose Page.setLifecycleEventsEnabled {} +0ms
method => browser:verbose Emulation.setScriptExecutionDisabled {"value":false} +0ms
method <= browser OK:verbose Emulation.setScriptExecutionDisabled {} +0ms
method => browser:verbose Page.navigate {"url":"https://www.lemonde.fr/"} +0ms
listen for event =>:verbose Page.lifecycleEvent +1ms
listen once for event =>:verbose Page.loadEventFired +0ms
listen once for event =>:verbose Page.domContentEventFired +0ms
listen for event =>:verbose Security.securityStateChanged +0ms
Driver:verbose Security.enable +1ms
method => browser:verbose Security.enable +0ms
<= event:verbose Security.securityStateChanged {"securityState":"neutral","schemeIsCryptographic":false,"explanations":[],"insecureContentStatus":{"ranMixedContent":false,"displayedMixedCon +0ms
method <= browser OK:verbose Security.enable {} +0ms
<= event:verbose Network.requestWillBeSent {"requestId":"45312F4011F5F2E5349C75E72FF949E7","loaderId":"45312F4011F5F2E5349C75E72FF949E7","documentURL":"https://www.lemonde.fr/","request":{" +1ms
Driver:verbose Waiting on DomContentLoaded +1ms
NetworkRecorder:verbose network semi-quiet +1ms
<= event:verbose Network.responseReceived {"requestId":"45312F4011F5F2E5349C75E72FF949E7","loaderId":"45312F4011F5F2E5349C75E72FF949E7","timestamp":103107.807272,"type":"Document","response +30ms
method <= browser OK:verbose Page.navigate {"frameId":"01E01B67E93DDA6A4FB4F159E8BEE17D","loaderId":"45312F4011F5F2E5349C75E72FF949E7"} +3ms
<= event:verbose Page.frameStartedLoading {"frameId":"01E01B67E93DDA6A4FB4F159E8BEE17D"} +4ms
<= event:verbose Page.lifecycleEvent {"frameId":"01E01B67E93DDA6A4FB4F159E8BEE17D","loaderId":"45312F4011F5F2E5349C75E72FF949E7","name":"init","timestamp":103107.814039} +1ms
<= event:verbose Log.entryAdded {"entry":{"source":"network","level":"error","text":"Failed to load resource: the server responded with a status of 521 ()","timestamp":1550766852464.34,"url +0ms
<= event:verbose Security.securityStateChanged {"securityState":"insecure","schemeIsCryptographic":true,"explanations":[{"securityState":"insecure","title":"Certificate","summary":"missing" +10ms
Driver:verbose Security.disable +0ms
method => browser:verbose Security.disable +0ms
status Disconnecting from browser... +2ms
Insecure document request: The URL you have provided does not have valid security credentials. This site is missing a valid, trusted certificate (net::ERR_CERT_COMMON_NAME_INVALID).
Lighthouse should launch the report.
Related issues
We bail completely if we see an insecure security certificate. There is some discussion happening to disable this behavior for sites hosted locally - see #6655.
I've never seen WPT used this way, very cool that it's possible to route requests for a domain to somewhere else. I assume this is a staging environment you are wanting to audit?
Until we work out what we want to do re: #6655, I think your best bet would be to host your staging environment on a subdomain (or even a different domain), one that has a valid security certificate. As a bonus, you wouldn't need muck with the DNS at all (if your site was https://staging.lemonde.fr, you could just audit that). If that wouldn't be viable please share why, would like to better understand your constraints.
note: unfortunately the --ignore-certificate-errors option only prevents certain feature gating (disallowing WebSockets on insecure pages) and showing the warning interstitial page. It doesn't change how consumer of the Devtools Protocol (like, Lighthouse) views security state.
Our use case is to to audit the website through a front end optimizer (Fasterize in our case) that rewrites pages to apply different front end optimizations.
This permits to validate the gain between the original website and the optimized website.
The audit is performed before actually plugging the website on the optmizer. At the time of the audit, the certificate is not installed on the platform.
I'm also having problems running Lighthouse with invalid certificates.
Use Case:
We have some internal environments where we deploy our app to, which don't have valid certificates. Normally, we just add exceptions in the browser and then be done with it. However, now we are running performance audits using Sitespeed.io with the Lighthouse plugin. This runs in a Docker container.
One solution might be to inject a step beforehand, to copy the certificates over to the store in the container, but that's a bit of a hassle.
I've obviously tried setting --ignore-certificate-errors, but as mentioned above, it doesn't alter the security state, so will still appear as insecure to the client.
I later tried going more low level to the Devtools protocol, and set the Security.setIgnoreCertificateErrors flag before running Lighthouse:
const client = await CDP({ port: chrome.port });
const { Security } = client;
await Security.enable();
await Security.setOverrideCertificateErrors({ override: true });
My original assumption was this would change the security state of Chrome and Lighthouse would use that. But now I'm thinking that this state is lost when Lighthouse connects as a new Devtools client, so must be done within Lighthouse itself.
Just in case this is helpful to others, my solution to this problem was copying certificate to the container so that the web app appears secure to the Lighthouse client. The two steps are as follows:
1) Copy the certificate to from the host machine to the container using docker cp
2) Run a startup script on the container that calls the certutil utility to populate the certificate store
Example:
certutil -d sql:$HOME/.pki/nssdb -A -t TC -n "My certificate" -i /my-cert.cer
I managed to work around this issue by installing an older version of lighthouse, in my case I did
npm install [email protected]
which you can then launch with
node_modules/lighthouse/lighthouse-cli/index.js --chrome-flags="--ignore-certificate-errors" --view <url>
that finished without throwing the ERR_CERT_COMMON_NAME_INVALID error.
We may merge a feature that would resolve this. see #7574.
With #8865, --chrome-flags="--ignore-certificate-errors" now works on master and will be in the next release. See this expired.badssl.com result as an example of the output.
Sorry for the noise, we thought --ignore-certificate-errors had been removed from Chrome because we can't find the impl and it's gone from the list, but this still seems to work as intended for now...
Most helpful comment
We may merge a feature that would resolve this. see #7574.