Foundation.mozilla.org: `Fixed blog page` cypress test is timing out

[patjouk]

Describe the bug
Percy tests are erroring because the test Fixed blog page is timing out.

Travis:

Percy:

@Pomax Can you take a look at what is happening there?
Meanwhile, I'm adding a skip to the broken test so that we can use Percy again.

[mmmavis]

I'm assigning this to myself since it's related to a ticket (add more visual regression tests) I'm working on this sprint.

[mmmavis]

@patjouk @Pomax

Okay I've found the cause. It's related to video streamfield having issues when the embed url provided to it isn't valid.

• In blog.py, we config it so that fake blog post page contains a video streamfield.
• But in streamfiled_provider.py, the url we used to create the fake video streamfield isn't a valid embed url... that's why it took Percy forever to wait for the embed video to load.

I've tested my theory in this PR by plugging in a real, working YouTube embed url... and Percy is no longer choking on it.

I wonder if we should upload something to s3 and just use that as the video we use in video factory?

[Pomax]

Hm, that's odd though, because cypress shouldn't care about whether an iframe loads or not, it should just take as screenshot no matter what the page looks like that that point. We might want to find out why it's refusing to (maybe we forgot a setting in Cypress that says "if you see a load error for anything, ignore that entire"?)

[mmmavis]

Um I've spent quite a bit of time on this ticket but am now hitting a wall... I've also gone through Cypress and Percy docs but couldn't find anything relevant regarding iframe timeout.

I don't think I have the bandwidth to spend on this ticket this milestone anymore. If we want to enable Percy test for the blog post page, the quickest solution is to make sure blog post page doesn't include a video streamfield (which creates <iframe>) to avoid the bad embed url timeout issue. Or we can remove this ticket from this milestone and find time to figure out how we wanna deal with iframe loading for Cypress/Percy tests.

Thoughts? @alanmoo @Pomax

[Pomax]

let's rip out the iframe, make the test work, and then have a ticket to put the iframe back in tied to a request for the percy folks to help us on this one using a normal support request on their side.

[Robdel12]

Hey everyone! Percy support here. What @mmmavis is saying is correct:

But in streamfiled_provider.py, the url we used to create the fake video streamfield isn't a valid embed url... that's why it took Percy forever to wait for the embed video to load.

Hm, that's odd though, because cypress shouldn't care about whether an iframe loads or not, it should just take as screenshot no matter what the page looks like that that point.

Percy isn't capturing _screenshots_ when the Cypress tests are running. We're capturing a DOM snapshot & then loading that DOM in a Puppeteer browser for "asset discovery". If the iframe doesn't have a valid src, this will cause both asset discovery and our rendering infrastructure to timeout. The timeout errors you're seeing in this screenshot are from Puppeteer failing to navigate to this DOM snapshot that was captured:

This is what it looks like when it fails to render in our browsers for a screenshot:

I wonder if we should upload something to s3 and just use that as the video we use in video factory?

This suggestion is spot on and I would highly recommend that approach. For Percy build # 3641, this is what the page looks like to us:

The URL https://robinson-sanchez.com/ is what is causing the issues here.

Edit: I've noticed in the time it took to write that comment, the emebded iframe seems to poll too, which is what is probably causing the timeout (repeated requests that hang & 522)

[Pomax]

@Robdel12 thanks for jumping in!

@cadecairos do we have a safe-but-generic video post on an S3 url that we can use as stable embed for testing?

[Pomax]

Some alternatives we might also look at:

• using an empty href="", although I don't know if that makes the iframe "work" or whether that makes it a passive element until it gets an href set (like <img> with a missing src)
• using a data-uri or blob: url, with stubbed HTML content
• using a //:0 url

That first one would certainly be _by far_ the easiest solution if it works, and the last two _might_ run into CSP limitations, but still worth looking at if the first idea doesn't work.

[mmmavis]

Thank you @Pomax and @Robdel12 for chiming in and providing help! ✨

[alanmoo]

Maybe we should upload the creative commons trailer for this project like the video player I interned at years ago used: https://peach.blender.org/about/

[Pomax]

honestly: we don't actually _care_ about what loads inside the iframe, just that it gets sized/placed correctly, so if we can update the test so that it doesn't even rely on the rest of the internet, that's probably better than uploading special pages somewhere.

But if we can't, CC for the win.

[mmmavis]

Woohooo passing an empty string as embed url works! PR on the way!

[Robdel12]

Anytime! Feel free to tag me in anything that comes up in the future too, I'd love to help 😃

[mmmavis]

PR merged! Thanks all!