Sitespeed.io: setup & teardown inside a script error:urlOrAlias.startsWith is not a function

[bodtx]

Hi when running a perf test separatly with a --preScript for the login it is working fine. But when I try to do one script with a setup method for login, it fails.

• Exact how you run (the exact parameters)
  docker run --rm -v "${PWD}:/sitespeed.io" artifactory.xxx.fr/xxx/sitespeedio/sitespeed.io:13.3.0 -n 1 ./performance/page-orga.js --multi --browsertime.login="xxx" --browsertime.password="password" --browsertime.url.root="https://xxxx" --browsertime.url.auth="https://xxxx"

• Which version you are using
  13.3.0

• OS (if you use Docker, report the base OS)
  windows 10

My script code:

async function setUp(context, commands) {
    await commands.navigate(context.options.url.auth );

      try {
        await commands.addText.byId(context.options.login, 'utilisateur_nom');
        await commands.addText.byId(context.options.password, 'mot_passe');
        await commands.click.byXpath('//*[@name="submit"]');
        return commands.wait.byXpath('//*[@title="Se déconnecter"]', 10000);
      } catch (e) {
        throw e;
      }
}

async function perfTest(context, commands) {
    // page testée
      await commands.measure.start('affiche liste dossier');
    await commands.navigate( context.options.url.root+'/organisme/?dummy2' );
    await  commands.measure.stop();
    return   commands.wait.byXpath('//*[@id="liste"]', 10000); 


};

async function tearDown(context, commands) {
    await commands.navigate( context.options.url.auth );
}

module.exports = {
          setUp: setUp,
          tearDown: tearDown,
          test: perfTest
        };

error:

[2020-06-26 14:20:19] ERROR: TypeError: urlOrAlias.startsWith is not a function
    at Measure.start (/usr/src/app/node_modules/browsertime/lib/core/engine/command/measure.js:141:19)
    at /usr/src/app/node_modules/browsertime/lib/core/engine/run.js:9:32
    at Iteration.run (/usr/src/app/node_modules/browsertime/lib/core/engine/iteration.js:192:15)
    at async Engine.runByScript (/usr/src/app/node_modules/browsertime/lib/core/engine/index.js:174:20)
    at async Object.analyzeUrl (/usr/src/app/lib/plugins/browsertime/analyzer.js:118:21)
    at async Object.processMessage (/usr/src/app/lib/plugins/browsertime/index.js:134:24)
[2020-06-26 14:20:19] ERROR: No data to collect
[2020-06-26 14:20:19] ERROR: TypeError: Cannot read property 'alias' of undefined
    at Object.processMessage (/usr/src/app/lib/plugins/browsertime/index.js:149:50)

[soulgalore]

Hi @bodtx thanks for the bug report. I'll have a look tonight and try to fix it (next release coming in a couple of days).

[soulgalore]

I simplified it and it works directly in Browsertime but not in sitespeed.io:

async function setUp(context, commands) {}

async function perfTest(context, commands) {
  // page testée
  await commands.measure.start('affiche liste dossier');
  await commands.navigate('https://www.sitespeed.io');
  return commands.measure.stop();
  //    return   commands.wait.byXpath('//*[@id="liste"]', 10000);
}

async function tearDown(context, commands) {}

module.exports = {
  setUp: setUp,
  tearDown: tearDown,
  test: perfTest
};

I think the problem could be that I've missed making sure exported scripts from one file works in sitespeed.io.

[soulgalore]

Yep the problem is that the functionality is only implemented in the CLI part of browsertime.

[soulgalore]

This is released, @bodtx can you let me know if it works for you?

[bodtx]

yes working great thx.
But I have maybe missunderstood I have 3 scripts .js, I use --multi to run them
when running the scripts are launched in the same time in the log, the setup login sequence is executed in multiple browsers:

[2020-07-20 09:32:59] INFO: Navigating to url https://dauth.xxxx iteration 1
[2020-07-20 09:33:08] INFO: Navigating to url https://dauth.xxxx iteration 1
[2020-07-20 09:33:16] INFO: Navigating to url https://dauth.xxxx iteration 1

I thought that each js journey would have been executed one after the others, as cookies are shared I cannot use different users I suppose

[soulgalore]

Hi @bodtx yes maybe the "multi" name isn't so good. Multi in this way means running multiple script after each other, not at the same time. I'll update the documentation so that is easier to understand.

[bodtx]

yes but if you look at the times in my previsou log it seems not to be the case, everything seems to be scrambles.
this log is done in the setup script of each .js file, so it should not be executed 3 at the same time, no?

INFO: Navigating to url https://dauth.xxxx iteration 1

[soulgalore]

I don't know, do you have a simplified use case I can test?

[bodtx]

ok sorry for the delay
the cli:
docker run --rm -v "${PWD}:/sitespeed.io" xxxxx/sitespeedio/sitespeed.io:14.1.0 -n 1 iteration1.js iteration2.js --multi

the two files
performance.zip

we can see in the log that the execution is not sequential:

Running tests using Chrome - 1 iteration(s)
Navigating to url https://www.baeldung.com/ iteration 1
Navigating to url https://www.baeldung.com/ iteration 1
Start to measure spring1
Navigating to url https://spring.io/ iteration 1
Start to measure spring2
Navigating to url https://spring.io/ iteration 1
Navigating to url https://stackoverflow.com/ iteration 1
Navigating to url https://stackoverflow.com/ iteration 1

expected

Running tests using Chrome - 1 iteration(s)
Navigating to url https://www.baeldung.com/ iteration 1
Start to measure spring1
Navigating to url https://spring.io/ iteration 1
Navigating to url https://stackoverflow.com/ iteration 1

Navigating to url https://www.baeldung.com/ iteration 1
Start to measure spring2
Navigating to url https://spring.io/ iteration 1
Navigating to url https://stackoverflow.com/ iteration 1

[soulgalore]

If you runtest1.js test2.js --multi -n 3 then test1.js will run, then test2.js in the same iteration (at least now when I tried), so I think this can have been fixed.