Sitespeed.io: Prescript does not work properly under hash route

[Nuoky]

Hello,

I use preScript to complete the login operation, But if the test target switches routes through hash change and login page and test page are in the same domain, the system cannot jump to the test page after running Prescript.

The selenium script is working properly under Sitespeed 6.3.5.

Version: Sitespeed 7.2.0

Docker:

docker run --privileged --shm-size=1g --rm -v "$(pwd)":/sitespeed.io -p 5766:5900 -v /etc/localtime:/etc/localtime:ro sitespeedio/sitespeed.io:7.2.0 'http://xxxxxx' --video --speedIndex --browsertime.preScript ./selenium/tt_login.js --browsertime.chrome.args no-sandbox --browsertime.pageLoadStrategy normal --browsertime.block --plugins.load analysisstorer --browsertime.viewPort "1360x768" -n 1 -b chrome

Selenium script:

module.exports = {
    run(context) {
        return context.runWithDriver(async function(driver){
            const webdriver = context.webdriver;
            const until = webdriver.until;
            const By = webdriver.By;

            // config
            const url = 'http://xxxxxxx';
            const userXpath = '//*[@id="UserName"]';
            const passwordXpath = '//*[@id="Password"]';
            const userName = 'xxxxx';
            const password = 'xxxxx';
            const clickXpath = '//*[@id="Submit"]';

            // run
            async function doLogin(url, userXpath, passwordXpath, clickXpath, userName, password) {
                await driver.get(url);
                await driver.wait(until.elementLocated(By.xpath(userXpath)));
        await driver.wait(until.elementLocated(By.xpath(passwordXpath)));
        await driver.wait(until.elementLocated(By.xpath(clickXpath)));
        await driver.sleep(2000);
                await driver.findElement(By.xpath(userXpath)).sendKeys(userName);
                await driver.findElement(By.xpath(passwordXpath)).sendKeys(password);
                await driver.findElement(By.xpath(clickXpath)).click();
                await driver.sleep(5000);
            }
            await doLogin(url, userXpath, passwordXpath, clickXpath, userName, password)
                .then(_ => console.log('SUCCESS!'), e => console.error('FAILURE: ' + e));
        });
    }
};

Sorry for my bad English.

[Nuoky]

@soulgalore
Hi,

https://github.com/sitespeedio/browsertime/commit/d7fefe3984b9d598087994d8ba11cde9d06910b7

I think this is the commit, changing

await driver.get(url);

to

document.body.innerHTML = '';
document.body.style.background = '#FFFFFF';
window.requestAnimationFrame(function(){
window.requestAnimationFrame(function(){
  window.location="${url}";
});
});

causes an error in the page that jumps to the hash route.

You can reproduce the problem by doing the following:

1) Chrome Browser opens https://preview.pro.ant.design/#/dashboard/analysis

2) Executed in Chrome DevTools:

document.body.innerHTML = '';
document.body.style.background = '#FFFFFF';
window.requestAnimationFrame(function(){
window.requestAnimationFrame(function(){
  window.location="https://preview.pro.ant.design/#/dashboard/monitor";
});
});

[Nuoky]

document.body.innerHTML = ''; 

This code removes the root node, causing single-page applications like React and Vue to not work properly.

[soulgalore]

HI @Nuoky

This code removes the root node,

ok, hmm, we use that to make sure that the background is empty and white (so the video is easy to extract), but we could change that to be a layer on top of all other things on the screen. Let me test the coming days how it all works out.

Best
Peter

[soulgalore]

Yeah ok, so if I understand correctly we introduced in 7 but it was only working in 6.x if you wasn't using video, is that right @Nuoky ? I think we always used that approach for video. I can make a fix the coming days but need your help @Nuoky to try it out.

[soulgalore]

One more thing, what kind of error do you get @Nuoky ? Is it #2087 ?

[Nuoky]

Sorry, I don't quite understand it. I can use video and prescript when I use 6.x. However, as long as the prescript is used in 7.2.0, the browser will be stuck in white screen (observed by x11vnc). I also tried to close video (--video false --browsertime.visualMetrics false) and the same problem will occur.

[Nuoky]

@soulgalore
Is the same error

[2018-07-10 14:12:38] ERROR: TypeError: Cannot read property 'id' of undefined
    at Object.getResultLogLine (/usr/src/app/node_modules/browsertime/lib/support/util.js:41:44)
    at Engine.run (/usr/src/app/node_modules/browsertime/lib/core/engine/index.js:114:19)
    at <anonymous>
From previous event:
    at queue.process.message (/usr/src/app/lib/core/queueHandler.js:208:17)
    at drainItem (/usr/src/app/node_modules/concurrent-queue/index.js:96:21)
    at Immediate.drain [as _onImmediate] (/usr/src/app/node_modules/concurrent-queue/index.js:77:84)
    at runCallback (timers.js:794:20)
    at tryOnImmediate (timers.js:752:5)
    at processImmediate [as _immediateCallback] (timers.js:729:5)

[soulgalore]

Ah sorry I wasn't being clear plus I was being wrong :)

What I meant was that before it worked like this:

• the pre script accessed one URL.
• then we navigate to the new URL (=writing the URL in the browser window and hit return).
  That generated a new request and everything work fine. But you tested doing a new navigation.

I've added a meta task (#2094) for tracking what we need to fix.

[soulgalore]

Just want to say I haven't forgotten about this, but need to sync with @tobli so we can discuss how to move on.

[tangyiming]

@soulgalore hope hash routes can be tested by sitespeed in next update

[soulgalore]

Yep lets see when we can make #2094 happen, I want to spend some time with @tobli first before we do any changes though, see https://github.com/sitespeedio/sitespeed.io#vacation-july---august

[Giners]

Hi @soulgalore,

Just learned about sitespeed and am really excited to try to get it working. I believe I am running into this problem. I just wanted to:
1) Confirm I am having the same problem
2) If so, when is an ETA on a fix

In terms of confirmation Im looking to test a page with a URL of https://mysite.com/index.html#/foo/bar and am trying to login via a pre-script that accesses the login page at https://mysite.com/index.html#/pages/signin. I get the same error of TypeError: Cannot read property 'id' of undefined. I presume this is the same problem? Sorry for a silly question but new to the area of site testing and this product and just want to double check.

I'm using:

• sitespeed.io: 7.4.0
• browsertime: 3.4.1

If it isn't the same error might you give me some ideas for starting points in how to troubleshoot? If it is the same error is there an ETA on when this might be fixed?

[soulgalore]

Hi @Giners perfect, it's always good to confirm (and it helps us too).

Confirm I am having the same problem

Yep it looks like that. The error is fixed in latest Browsertime but we haven't released sitespeed.io yet. Or it's actually two problems: 1. The error you got (that is fixed in Browsertime) 2. In Chrome navigating in Chrome in some circumstances doesn't generates a new page in the HAR file (that is not fixed in master release but I want to give it a shot before next release).

I'm working on collecting a couple of fixes to roll out a bigger release. It would take a couple of weeks until 7.5.0 is there. But if I remember correctly this will work fine in Firefox, can you try that for now -b firefox? Please let me know if it works.

Best
Peter

[Giners]

Using the -b firefox option worked. Thanks Peter.

We are going to look to start measuring our production environment with this. We will focus on using Firefox while we get acclimated to sitespeed.io and be on the look out for the 7.5.0 release.

[soulgalore]

Great! I will have a look this weekend and see if I can fix it for Chrome too.

[soulgalore]

Also @Giners if you could share two URLs that are public (and not behind any login) that would help me so I can make sure it works as it should peter [AT] soulgalore [dot] com.

Thanks :)

[soulgalore]

Hey @Nuoky sorry for taking so long time to fix this. If I understand correctly Chrome has internally changed the events we use to create the HAR file so in Chrome 69 this seems to work (as long as we run the latest Browsertime that we merged into master).

Any change you have the time to help me to see if this works for you now? You need to use the latest build of our Docker container: sitespeedio/sitespeed.io-autobuild:latest

Let me know if you get the chance!
Thanks
Peter

[soulgalore]

This will roll out in 7.5.0 later this week, please let me know if it works :)

[Nuoky]

Hi @soulgalore

I tried 7.5.0, there was no previous error, but another error occurred.

[2018-10-10 17:07:19] INFO: 0, backEndTime: 24ms, firstPaint: 162ms, firstVisualChange: 0ms, DOMContentLoaded: 421ms, Load: 2.79s, speedIndex: 0, visualComplete85: 0ms, lastVisualChange: 0ms, rumSpeedIndex: 448
[2018-10-10 17:07:19] ERROR: TypeError: Cannot read property '_meta' of undefined
    at Object.processMessage (/usr/src/app/lib/plugins/browsertime/index.js:137:58)
    at <anonymous>
From previous event:
    at queue.process.message (/usr/src/app/lib/core/queueHandler.js:208:17)
    at drainItem (/usr/src/app/node_modules/concurrent-queue/index.js:96:21)
    at Immediate.drain [as _onImmediate] (/usr/src/app/node_modules/concurrent-queue/index.js:77:84)
    at runCallback (timers.js:810:20)
    at tryOnImmediate (timers.js:768:5)
    at processImmediate [as _immediateCallback] (timers.js:745:5)

[soulgalore]

Hi @Nuoky thanks for getting back. I've fixed so it works with a preURL but then it's still a problem with preScript. I can also see that the firstVisualChange is wrong in your log, so there are more things that aren't working. Any chance you can share a full use case with me so that I can reproduce and fix it? peter [at] soulgalore [dot] com.

Best
Peter

[Nuoky]

Hi @soulgalore

The website I tested is not publicly available on the internet, so use other websites here as an example.

./selenium/hackernews.js

module.exports = {
    run(context) {
        return context.runWithDriver(async function(driver){
            const webdriver = context.webdriver;
            const until = webdriver.until;
            const By = webdriver.By;

            // config
            const url = 'https://dvajs.github.io/dva-hackernews/#/top';

            // run
            async function doLogin(url) {
                await driver.get(url);
                await driver.sleep(5000);
            }
            await doLogin(url)
        });
    }
};

run

docker run -it --privileged --shm-size=1g --rm -v "$(pwd)":/sitespeed.io -p 5766:5900 -v /etc/localtime:/etc/localtime:ro sitespeedio/sitespeed.io:7.5.1 'https://dvajs.github.io/dva-hackernews/#/show' --debug --video true --browsertime.visualMetrics true --browsertime.preScript ./selenium/hackernews.js --browsertime.chrome.args no-sandbox --browsertime.block -n 1 -b chrome

output

Google Chrome 69.0.3497.100 
Mozilla Firefox 62.0
[2018-10-11 17:45:19] INFO: Versions OS: linux 4.13.0-45-generic nodejs: v8.12.0 sitespeed.io: 7.5.1 browsertime: 3.7.2 coach: 2.3.0
[2018-10-11 17:45:19] INFO: {
  "uuid": "594054f5-b4ea-4a6e-af5c-52fb88eb5b5d",
  "type": "sitespeedio.setup",
  "timestamp": "2018-10-11T17:45:19+08:00",
  "source": "queueHandler"
}
[2018-10-11 17:45:19] INFO: {
  "uuid": "c298b71f-f9da-4478-8070-5d1bdb7cada8",
  "type": "browsertime.setup",
  "timestamp": "2018-10-11T17:45:19+08:00",
  "source": "browsertime"
}
[2018-10-11 17:45:19] INFO: {
  "uuid": "6988f6eb-bb0f-4fd1-b585-9c39bd6b01bd",
  "type": "browsertime.config",
  "timestamp": "2018-10-11T17:45:19+08:00",
  "source": "browsertime",
  "data": "{...}"
}
[2018-10-11 17:45:19] INFO: {
  "uuid": "3606aba7-0bf8-4d31-9c43-ae56c2df52e7",
  "type": "browsertime.config",
  "timestamp": "2018-10-11T17:45:19+08:00",
  "source": "coach",
  "data": "{...}"
}
[2018-10-11 17:45:19] INFO: {
  "uuid": "5a946937-a6f2-4a29-a024-ff3d222b2769",
  "type": "url",
  "timestamp": "2018-10-11T17:45:19+08:00",
  "source": "url-reader",
  "data": {},
  "url": "https://dvajs.github.io/dva-hackernews/#/show",
  "group": "dvajs.github.io"
}
[2018-10-11 17:45:19] INFO: Starting chrome for analysing https://dvajs.github.io/dva-hackernews/#/show 1 time(s)
[2018-10-11 17:45:19] INFO: Testing url https://dvajs.github.io/dva-hackernews/#/show iteration 1
[2018-10-11 17:45:37] INFO: 0, backEndTime: 944ms, firstPaint: 1.52s, firstVisualChange: 0ms, DOMContentLoaded: 2.77s, Load: 3.17s, speedIndex: 0, visualComplete85: 0ms, lastVisualChange: 0ms, rumSpeedIndex: 1531
[2018-10-11 17:45:37] ERROR: TypeError: Cannot read property '_meta' of undefined
    at Object.processMessage (/usr/src/app/lib/plugins/browsertime/index.js:137:58)
    at <anonymous>
From previous event:
    at queue.process.message (/usr/src/app/lib/core/queueHandler.js:208:17)
    at drainItem (/usr/src/app/node_modules/concurrent-queue/index.js:96:21)
    at Immediate.drain [as _onImmediate] (/usr/src/app/node_modules/concurrent-queue/index.js:77:84)
    at runCallback (timers.js:810:20)
    at tryOnImmediate (timers.js:768:5)
    at processImmediate [as _immediateCallback] (timers.js:745:5)
[2018-10-11 17:45:37] INFO: {
  "uuid": "c3b8f493-67b4-4f35-9dd6-4543e5d26923",
  "type": "sitespeedio.summarize",
  "timestamp": "2018-10-11T17:45:37+08:00",
  "source": "queueHandler"
}
[2018-10-11 17:45:37] INFO: {
  "uuid": "aee78680-2ab6-4dff-8320-60c41a2cf794",
  "type": "aggregateassets.summary",
  "timestamp": "2018-10-11T17:45:37+08:00",
  "source": "assets",
  "data": {},
  "group": "total"
}
[2018-10-11 17:45:37] INFO: {
  "uuid": "353751ff-46db-4c66-b759-ff7c0bfed6cf",
  "type": "largestassets.summary",
  "timestamp": "2018-10-11T17:45:37+08:00",
  "source": "assets",
  "data": {},
  "group": "total"
}
[2018-10-11 17:45:37] INFO: {
  "uuid": "7e2f2108-b183-4fc0-9040-e569c6235fd1",
  "type": "slowestassets.summary",
  "timestamp": "2018-10-11T17:45:37+08:00",
  "source": "assets",
  "data": {},
  "group": "total"
}
[2018-10-11 17:45:37] INFO: {
  "uuid": "9b48c708-a9d8-480b-95cd-092b33ce4eef",
  "type": "domains.summary",
  "timestamp": "2018-10-11T17:45:37+08:00",
  "source": "domains",
  "data": {},
  "group": "total"
}
[2018-10-11 17:45:37] INFO: {
  "uuid": "cba26354-a118-4771-a89d-9f81974cbf89",
  "type": "sitespeedio.render",
  "timestamp": "2018-10-11T17:45:37+08:00",
  "source": "queueHandler"
}
[2018-10-11 17:45:37] INFO: HTML stored in /sitespeed.io/sitespeed-result/dvajs.github.io/2018-10-11-17-45-19
[2018-10-11 17:45:37] INFO: {
  "uuid": "4c3d4a31-5fbd-41f7-a49c-f569fc061812",
  "type": "html.finished",
  "timestamp": "2018-10-11T17:45:37+08:00",
  "source": "html"
}
[2018-10-11 17:45:37] INFO: Finished analysing https://dvajs.github.io/dva-hackernews/#/show

[soulgalore]

Thanks @Nuoky !! Thanks again for taking the time to make a reproducible case, I'll look into it asap.

Best
Peter

[soulgalore]

This is the minimal to reproduce in Browsertime:
docker run --shm-size=1g --rm -v "$(pwd)":/browsertime sitespeedio/browsertime 'https://dvajs.github.io/dva-hackernews/#/show' --preScript ./hackernews.js -n 1 -b chrome --videoParams.keepOriginalVideo --chrome.collectPerfLog

[soulgalore]

@Nuoky Ok, I think I got a fix now. I've changed how we handle how we record the video and changed how we create the HAR from Chrome. I've pushed it to a branch in Browsertime, any chance you can try that with your real page so we know that it works before I move the code forward to a new release?

You can test it like this:
docker run --shm-size=1g --rm -v "$(pwd)":/browsertime sitespeedio/browsertime-autobuild:better-orange 'https://dvajs.github.io/dva-hackernews/#/show' --preScript ./hackernews.js -n 1 -b chrome --pageCompleteCheckInactivity

I've changed when to end the test since there's no onload event fired in the page when we do the next step. We can do some tuning here to make the "end" better, I need to look more into the code.

[Nuoky]

@soulgalore cooool, run successfully thx!

[soulgalore]

Great @Nuoky I'be just released the changes in 7.6.0, hope it works for real this time.

[Nuoky]

I tried 7.6.2 and I can successfully output the result.

[soulgalore]

Perfect, lets close this now and hope that it will continue to work :)

[HackXiong]

Hi soulgalore,
I tried 7.6.2 with multiple urls and there are 2 issues:

1. All the video is the same on each pages;
2. The

   title is the same(last one) on each pages.

Here is my result:
https://onevmw-my.sharepoint.com/:u:/g/personal/qinh_vmware_com/EVW3NUD47UJBlA6QcS0FMf8BImkt35ouwpSqfW-KHNrkvg?e=tFd5DP
Can you take look this? thanks.

[soulgalore]

Hi @HackXiong any chance you could open a separate issue + if you can share a little more on how you run your test so I best case can reproduce the issue?

Thanks
Peter

[soulgalore]

Hi again @HackXiong I've open https://github.com/sitespeedio/sitespeed.io/issues/2181 for you. Has this ever worked? If you add the new --useHash to your cli param, it will work. Or are you using that already?

Best
Peter