Sitespeed.io: Cannot get the metrics when using Google Chrome and added the measure commands start/stop for specific section in the script

[LuisOsv]

Hi all,

I have the following script, and I want to measure how much time it takes to show the context menu with the desired text.

module.exports = async function(context, commands) {
    await commands.navigate(
      'https://www.wikipedia.org/'
    );
    await commands.wait.byId('searchInput');
    await commands.measure.start('measure search');
    await commands.addText.byId('Software Testing', 'searchInput');
    await commands.wait.byXpath('//h3[contains(@class, "suggestion-title") and text() = "Software testing"]', 10000);
    return await commands.measure.stop();
};

And I'm using the following comand to run on docker container:
docker run --rm -v ${pwd}:/sitespeed.io sitespeedio/sitespeed.io /sitespeed.io/Wikipedia.js -b chrome --multi -n 1

Then I got the error message:

Google Chrome 84.0.4147.89 
Mozilla Firefox 79.0
[2020-07-30 01:33:03] INFO: Versions OS: linux 4.19.76-linuxkit nodejs: v12.18.2 sitespeed.io: 14.2.1 browsertime: 9.1.0 coach: 5.1.0
[2020-07-30 01:33:03] INFO: Running tests using Chrome - 1 iteration(s)
[2020-07-30 01:33:04] INFO: Navigating to url https://www.wikipedia.org/ iteration 1
[2020-07-30 01:33:11] INFO: Start to measure measure search
[2020-07-30 01:33:15] ERROR: Could not add meta data to the HAR, miss page 0
[2020-07-30 01:33:15] INFO: https://www.wikipedia.org/ TTFB: 1.13s, firstPaint: 1.42s, firstVisualChange: 100ms, FCP: 1.42s, DOMContentLoaded: 1.60s, LCP: 1.58s, CLS: 0, Load: 1.61s, speedIndex: 431ms, perceptualSpeedIndex: 114ms, contentfulSpeedIndex: 100ms, visualComplete85: 468ms, lastVisualChange: 468ms
[2020-07-30 01:33:15] ERROR: Could not find the right index 0 for har for url https://www.wikipedia.org/ 
[2020-07-30 01:33:15] ERROR: Error: PageIndex out of range
    at module.exports.pickAPage (/usr/src/app/node_modules/coach-core/lib/har/harCutter.js:18:11)        
    at Object.pickAPage (/usr/src/app/node_modules/coach-core/lib/index.js:83:12)
    at Object.processMessage (/usr/src/app/lib/plugins/browsertime/index.js:229:29)
[2020-07-30 01:33:16] INFO: HTML stored in /sitespeed.io/sitespeed-result/Wikipedia_js/2020-07-30-01-33-03

However, running for Firefox it works fine:
docker run --rm -v ${pwd}:/sitespeed.io sitespeedio/sitespeed.io /sitespeed.io/Wikipedia.js -b firefox --multi -n 1

Best Regards

[soulgalore]

Hi @LuisOsv sorry for the late answer and thanks for taking the time to rewrite your problem with a public URL.

I'm does it really work on Firefox or is it just an error for Chrome? I think when you want to measure something else than a navigation, you need choose what metric to focus on. I think in this case, the only visual metric that could pickup when you add things to the page is LastVisualChange (as long as you don't implement your own user timing).

To get that to work in both browsers I would change the order and thinking when you start the test doesn't matter since the only metric here that measure the search is last visual change:

    await commands.measure.start('measure search');
    await commands.navigate(
      'https://www.wikipedia.org/'
    );
    await commands.wait.byId('searchInput'); 
    await commands.addText.byId('Software Testing', 'searchInput');
    await commands.wait.byXpath('//h3[contains(@class, "suggestion-title") and text() = "Software testing"]', 10000);
    return await commands.measure.stop();

But why did you get the error in Chrome? Maybe we can implement it smarter, but today we use the Chrome internal trace to generate the HAR file and right now that only works when you navigate to a page. For Firefox we use Mozillas HAR extension and I guess it have all the data needed even though we didn't start the navigation within the measure step.

In your case I would add a custom metric. If you own your site and can change the code, you can add a https://wicg.github.io/element-timing/ in that search box when its populated. If you don't own it, you can use --browsertime.scriptInput.visualElements - it's a way to pinpoint a element that will be analysed in the video so you can get your own metric for that.

Include specific elements in visual elements. Give the element a name and select it with document.body.querySelector. Use like this: --scriptInput.visualElements name:domSelector . Add multiple instances to measure multiple elements. Visual Metrics will use these elements and calculate when they are visible and fully rendered. I think we miss out on good docs for that, so please let me know if you need help and I can add a better example in the doc.

[LuisOsv]

@soulgalore Ok thanks for the response, and yes I think is better to have a good example in the documentation about to include custom metrics, include specific elements in visual elements.

[soulgalore]

Let me add an example tomorrow so we have that.

[LuisOsv]

hi @soulgalore did you add the example?

[soulgalore]

No sorry, I'm in the middle of rewriting visual metrics, but let me add an example tomorrow. We have some documentation about it at https://www.sitespeed.io/documentation/sitespeed.io/video/#collect-visual-elements-metrics but it would be nice to have an example.

[soulgalore]

@LuisOsv I've updated with an example: https://www.sitespeed.io/documentation/sitespeed.io/video/#collect-visual-elements-metrics

[LuisOsv]

Hi @soulgalore thanks for the adding a good example.

I have one question please, can I use the "scriptInput.visualElement" in my script?

[soulgalore]

@LuisOsv ah you mean you want to dynamic choose in scripting? No right now you need to get that before you do your run.