Cypress: Screenshots taken during 'cypress run' do not respect the viewport size set

[egucciar]

Current behavior:

Observe these two images:

Headless / Electron:

Headed / Chrome:

Notice the spacing to the Left of our Logo is much different than the spacing to left of the Logo in headed and headless, indicating that our 1500px viewport setting was not respected in headless.

Desired behavior:

Ideally, the Viewport width should be the same for both browsers, headless and headed.

Steps to reproduce:

Try using a Website with a viewport of 1500 pixels wide and confirm if the viewport taken in headless vs headed is the same

Versions

Cypress 3.0.2 and MacOS

Motivation

I am working towards pegging down the Visual Diffing workflow for our company. Truth is, having comparable screenshots in headless vs headed runs is a desirable feature. When it comes to scaling and cropping, there is no issue on my side to do some POC work to scale/crop images and set thresholds such that differences in the CI / Headless enviroment can be accounted for and ignored. However I cannot ignore the fact that headless screenshots do not honor the viewport width I explicitly set in the Cypress.json. Ideally, if the viewport is 1500px wide headed, the headless screenshots should reflect the same.

[Bkucera]

@egucciar have you tested multiple settings of the viewport width and confirmed that it is always ignored in headless electron?
Possibly related to #1857

[egucciar]

@Bkucera I agree that this needs more information. I tried to output the "innerWidth" of window in both scenarios and Headless also said 1500px. Maybe it's just a difference between how High DPI screens render UIs verses the headlesS? maybe this is not a bug then, it just is confusing to me why there would be such different margins in one vs the other.

[brian-mann]

I believe the reason this is happening is because in headless mode we set XVFB to render at 1280x720.

Normally Cypress scales the screen so that running tests makes no difference to the screen size. However when taking screenshots, we no longer scale the image, which forces it to display in full width + height.

Think of the 1280x720 as the literal screen size. If you've set your viewportWidth to be 1500px then it physically cannot fit inside of 1280. Your application is responsive so that it all "works" without a scrollbar, but its not the same image anymore.

We'd need to support setting xvfb viewport via the command-line, or we'd perhaps need to issue a warning or even error if you're trying to take a cy.screenshot() at a viewport that the screen cannot actually display.

[egucciar]

Wow! Awesome that's solved for me, there's a Max width so that's good. The 1500 wasn't super necessary I'll set to 1280.

It would be nice if full length screens could be taken by sizing up the viewport rather than scrolling due to fixed header content. I wonder if there has to be some kind of boundary on the width / height in general of the view box for taking shots or why it can't be any size?

[egucciar]

@brian-mann Seeing unreliable/undesirable results with the fullPage screenshot capture. It would be awesome if the xvfb could respect the same viewport set in the Cypress.json, or an option to force it to do so, so I could use viewport as the capture setting while also ensuring the viewport is indeed large enough to capture enough of the application I care about.

[ykravv]

Hey,

I was able to make screenshots in headless electron with setting resolution as:

// plugins/index.js 
module.exports = on => {
  on('before:browser:launch', browser => {
    if (browser.name === 'electron') {
      return {
        width: 1600,
        height: 900,
      };
    }
  });
};

But the question is how to get the same effect for Chrome? Trying to get it working on CI in headless mode.

So, my config has 1600x900 viewport size.
On CI I'm getting failing screenshots tests with resolution of screenshots - 1050x938
I tried to log screen.width/screen.height, and looks like screen resolution on CI - 1280x1024

On before:browser:launch I see chrome arg --start-maximized, seems it does not work at all?

I'm totally new to xvfb and how cypress work with it. Please point me to the right direction, is there any way to get desired resolution?

[ykravv]

Hey @brian-mann

We'd need to support setting xvfb viewport via the command-line

Is there any possibility to do that?

[jennifer-shehane]

You can pass a -screen option to xvfb-run specifying the resolution. See XVFB docs

Code where Cypress passes 1280x1024x8 option: https://github.com/cypress-io/cypress/blob/develop/packages/server/test/scripts/run.js#L55

−screen screennum WxHxD
This option creates screen screennum and sets its width, height, and depth to W, H, and D respectively. By default, only screen 0 exists and has the dimensions 1280x1024x8.

[ykravv]

Thanks @jennifer-shehane

I'm trying to get it working with docker image cypress/browsers:chrome67-ff57, in my docker file I have the following lines:

RUN Xvfb :1 -screen 0 '1600x1024x16'  &
RUN export DISPLAY=:1

Is that enough for cypress to pick up the display or am I missing something?
Update: Looks it does not work, getting cy.screenshot failing on timeout

[jennifer-shehane]

@YevheniiKravchenko Sorry, I was addressing the core changes needed for cypress to fix this issue.

To pass the options to chrome, you will have to pass the args slightly differently. Example of plugins file:

module.exports = (on, config) => {
  on('before:browser:launch', (browser = {}, args) => {
    if (browser.name === 'chrome') {
      args.push('--cast-initial-screen-width=1600')
      args.push('--cast-initial-screen-height=900')

      return args
    }

    if (browser.name === 'electron') {
      args.width = 1600
      args.height = 900

      return args
    }
  })
}

I didn't try this locally, so let me know if this works! Be sure to check our browser launch api docs.

[mgrybyk]

any updates on how to set screen size to desired value instead of hardcoded 1280x720 ?

[AndreVirtimo]

I had the same problem. Our CI server didn't made the screenshots in the right resolution.

I fixed it by changed the https://github.com/cypress-io/cypress/blob/develop/cli/lib/exec/xvfb.js and added xvfb_args. Here is the result: https://gist.github.com/AndreVirtimo/7bd2892478af2ea06ea7445319fbfef1#file-xvfb-js-L12-L16

I'm using hard coded FullHD resolution, but there should be used the viewPort configuration from cypress. I don't know how to get access to the configuration at this point.

The result are screenshots with the resolution 1919x1080. I don't now why the is a pixel missing. But for me this is ok at this point.

[lukeapage]

On windows I've had some success running this in headless mode before cypress:

https://gallery.technet.microsoft.com/scriptcenter/2a631d72-206d-4036-a3f2-2e150f297515

[infctr]

@jennifer-shehane Setting width and height for headless electron in docker container produces screenshots with correct size (1919x1080 actually), however I seem to get a video recording error.

// config
module.exports = (on, config) => {
  on('before:browser:launch', (browser = {}, args) => {
    if (browser.name === 'electron') {
      args.width = 1920;
      args.height = 1080;

      return args;
    }
  })
}

Warning: We failed to record the video.

This error will not alter the exit code.

Error: ffmpeg exited with code 1: Error initializing output stream 0:0 -- Error while opening encoder for output stream #0:0 - maybe incorrect parameters such as bit_rate, rate, width or height
Conversion failed!

    at ChildProcess.<anonymous> (/root/.cache/Cypress/3.2.0/Cypress/resources/app/packages/server/node_modules/fluent-ffmpeg/lib/processor.js:182:22)
    at emitTwo (events.js:125:13)
    at ChildProcess.emit (events.js:213:7)
    at Process.ChildProcess._handle.onexit (internal/child_process.js:200:12)

Upd:
This seems related https://github.com/cypress-io/cypress/issues/3491

Upd2:
My current workaround is to add an extra 1px to Xvfb display setup

#!/bin/sh
Xvfb :1 -screen 0 1921x1080x24 >/dev/null 2>&1 &
export DISPLAY=:1.0

yarn nps test.cy.ci

[timmydoza]

@infctr What docker image are you using? I can't get the args.width = 1400 trick to work. Also where are you changing the settings for xvfb? Is it in the dockerfile, or is the script run somewhere else?

[infctr]

@timmydoza I've extended the image from the official repo https://github.com/cypress-io/cypress-docker-images/blob/master/browsers/chrome69/Dockerfile, added latest yarn and dependencies needed for my project.

The script above runs on CI as a job, sets xvfb display settings and starts the tests. You can run it as an entrypoint for you docker container. Nothing really fancy here

[timmydoza]

Perfect - got it working. Thanks! Couldn't get it working with chrome, but it works with electron and the base:10 image.

[jennifer-shehane]

Related issue: https://github.com/cypress-io/cypress/issues/3324

[sebinsua]

What I've noticed is that require('electron').screen.getPrimaryDisplay() returns completely different width, height and scaleFactor depending on whether cypress run is run with the command line argument --headed or not.

When it is headless, it picks up the resolution of my monitor (2560 x 1440) and a scale factor of 1. When it is headed, it seems to have a scale factor of 1.2999999523162842 and has resolutions (1969 x 1107) that don't match the viewport I selected at all (1280 x 720). It then generates images which are of different sizes.

Edit: Internally BrowserPage#capturePage() seems to produce images of different sizes dependent on the Display (the width and height are both multiplied by the scaleFactor).

We would like to be able to compare screenshots that were generated in headed mode in headless mode (and vice versa). I am not sure whether it would be possible to ensure that screenshots are normalized, or whether the solution would be something else?

The problem with just using Xvfb to control the Display that is used, is that we can't do this in a headed mode while still keeping the ability to interact with the Cypress GUI.

Edit 2: These lines imply that the scaling should be 1 in headless mode. We need to be able to control the scaling factor so it is the same across environments (browsers, displays, headed, headless), otherwise image regressions will fail for environmental reasons.

Edit 3: This fixes my problem in everywhere but a headed electron browser.

cypress/plugins/index.js:

const {
  addMatchImageSnapshotPlugin
} = require('cypress-image-snapshot/plugin');

module.exports = (on, config) => {
  on('before:browser:launch', (browser = {}, args) => {
    if (browser.name === 'chrome' || browser.name === 'chromium') {
      // In headless mode, Cypress fixes the scale factor to 1, and this forces
      // screenshots to be taken with an image size matching the viewport size
      // instead of the viewport size multiplied by the scale factor.
      //
      // Since we also want to run the image regression tests in development mode,
      // we need to set the device scale factor to 1 in chrome / chromium.
      //
      // See: https://github.com/cypress-io/cypress/issues/2102#issuecomment-521299946
      // See: https://github.com/cypress-io/cypress/blame/a7dfda986531f9176468de4156e3f1215869c342/packages/server/lib/cypress.coffee#L132-L137
      args.push('--force-device-scale-factor=1');
    } else if (browser.name === 'electron' && browser.isHeaded) {
      // eslint-disable-next-line no-console
      console.log(
        "There isn't currently a way of setting the device scale factor in Cypress when running headed electron so we disable the image regression commands."
      );
    }

    return args;
  });

  addMatchImageSnapshotPlugin(on, config);
};

cypress/support/commands.js:

// See: https://docs.cypress.io/plugins/#visual-testing
import { addMatchImageSnapshotCommand } from 'cypress-image-snapshot/command';

if (Cypress.browser.name === 'electron' && Cypress.browser.isHeaded) {
  Cypress.Commands.add(
    'matchImageSnapshot',
    {
      prevSubject: ['optional', 'element', 'window', 'document']
    },
    (_, name) => {
      cy.log(
        "In non-headless electron we can't control the device scale factor so have made `CypressSubject#matchImageSnapshot` a noop."
      );
    }
  );
} else {
  addMatchImageSnapshotCommand();
}

[rwwagner90]

I'm not sure if the issues here are the same as the issues I am experiencing or not, but I have some tests where I need to check specific pixel values for translateX etc. Locally they all pass, but when running in CI, they are off by a few pixels. I was hoping to find a way to ensure all environments, both local and CI had the same dimensions and pixel density. Has anyone had success with this?

[ispal]

Hi,

With all these messages, I'm not fully sure that are you going to make xfvb args as public properties without the need to update them manually? In CI environment where node_modules folder is cleared before every build, it's not really usable currently.

[ispal]

Our CI is on a Windows machine and the screenshots taken are 1024x768 which are way too small. I checked the code and there's a check for Windows and those arguments are not passed on in Windows environment. It must behave differently in Windows? Anyway, where is that 1024x768 configured or is it some default value? Can one edit those values?

You can pass a -screen option to xvfb-run specifying the resolution. See XVFB docs

Code where Cypress passes 1280x1024x8 option: https://github.com/cypress-io/cypress/blob/develop/packages/server/test/scripts/run.js#L55

−screen screennum WxHxD
This option creates screen screennum and sets its width, height, and depth to W, H, and D respectively. By default, only screen 0 exists and has the dimensions 1280x1024x8.

[jennifer-shehane]

I made some comments on how screenshots sizes work some and how we will improve the documentation around it here: https://github.com/cypress-io/cypress/issues/3324#issuecomment-542414532 May be relevant.

[ispal]

Hmm..I think my issues is not related to that. The problem for us is that we get different screenshot sizes by running module version of Cypress headlessly so there's no cypress run vs cypress open

Here's the interesting part. If I remotely connect to a computer where our CI is, open cmd and run node cypress-runner manually, I will get the correct screenshot 1600x1000px, but If I run that command as part of build step, I will get a screenshot with size 1024x768px. So it's the same machine and script is run similarly, but results differ.

We have added the following to the plugins section and I have confirmed that in both circumstances it will go where those args are set.

on('before:browser:launch', (browser = {}, args) => {
    if (browser.name === 'electron') {
      args.width = 1600;
      args.height = 1000;

      return args;
    }
});

Do you have any pointers why that would happen? Could it be that if I RDP to that machine, it will get the screen of my client machine and that's why it works?

EDIT. I believe that 1024x768 is the default screen resolution on the CI machine and when I RDP, it will get my screen's resolution

[rwwagner90]

I think in my case, since the px were only off by a few, it is likely differences in scrollbar sizes in CI. Is there a way to account for that to test transform values with Cypress?

[lukeapage]

@ispal in headless ci in Windows you need to change the screen resolution. I do this : https://github.com/cypress-io/cypress/issues/2102#issuecomment-472745419

[tonilopezmr]

I'm not able to make screenshot testing with cypress working on CI, because, Image size (1280x1024) different than saved snapshot size 1279x1023.. I'm using Github Actions for that and I can't execute Xvfb command. Also, cypress runs an Xvfb immediately before to run the cypress.

Cypress config

{
"viewportWidth": 1280,
"viewportHeight": 1024
}

Cypress plugin setting width and height viewport.

const {
  addMatchImageSnapshotPlugin
} = require('cypress-image-snapshot/plugin');

module.exports = (on, config) => {
  addMatchImageSnapshotPlugin(on, config);
  on('before:browser:launch', (browser = {}, args) => {
    if (browser.name === 'electron') {
      args.width = 1280;
      args.height = 1024;

      return args;
    }
  });
};

[AndreVirtimo]

@tonilopezmr I'm change my "node_modules/cypress/lib/exec/xvfb.js" File directly and add xvfb args to set my needed screen size. Here is my script from our CI workflow:

sed -i "24 a xvfb_args: ['-screen','0','1920x1080x8']," node_modules/cypress/lib/exec/xvfb.js

[tonilopezmr]

@AndreVirtimo Sorry I don't understand your lines, I'm in the xvfb.js but which changes are you talking about? I'm don't see any xvfb args.

Edit:

I think you mean to add the xvfb_args on the Xvfb object like this?

var xvfb = Promise.promisifyAll(new Xvfb({
  xvfb_args: ['-screen','0','1280x1024x8'],
  timeout: 5000, // milliseconds
  onStderrData: function onStderrData(data) {
    if (debugXvfb.enabled) {
      debugXvfb(data.toString());
    }
  }
}));

[AndreVirtimo]

@tonilopezmr "sed" inserts a new line in this file. The result snipped is this:

(...)
module.exports = {
_debugXvfb: debugXvfb, // expose for testing
xvfb_args: ['-screen','0','1920x1080x8'],

_xvfb: xvfb, // expose for testing

start: function start() {
debug('Starting XVFB');
(...)

It seems that cypress does not set the xvfb screen size.

[tonilopezmr]

@AndreVirtimo And how did you run that on the CI?

I have tried to add these arguments but it still failing: https://github.com/Karumi/LitElementPlayground/runs/268134245

I'm copying xvfb.js file with the arguments into the node_modules xvfb.js running this script https://github.com/Karumi/LitElementPlayground/blob/add-cypress/cypress/scripts/xvfbargs.js.

And this is the cypress dashboard: https://dashboard.cypress.io/#/projects/144fs5/runs

Edit:

Here is on travis: configuration travis file

Same error on dashboard: https://dashboard.cypress.io/#/projects/144fs5/runs/14/specs

Now it seems if I add on travis the service: xvfb now the size is lower (1023x767)

Edit2:

It worked on travis!!! Editing the viewport for electron, and executing xvfb --screen adding one more pixel in width and height worked!!!!!

[ispal]

@ispal in headless ci in Windows you need to change the screen resolution. I do this : #2102 (comment)

Yeah, maybe that or I might give this a go: https://docs.microsoft.com/en-us/powershell/module/servercore/set-displayresolution?view=win10-ps

[sebinsua]

Since posting my last comment, I upgraded to Cypress 3.6 with headed Chrome 76 within a Docker container.

After doing this the maximum width of the browser in headless mode ended up being 1050px. That is despite the viewportWidth being set to 1280px. It appears to be clipped to this value, since by using Xvfb as others did above I could only force the width to be a number below 1050. I'm not sure why this is the case.

I've noticed that the number 1050 does appear in a number of issues where people are using --browser chrome in a Docker container, so I don't think I'm the only person to get this.

For now, the quick fix seems to be setting the viewportWidth to 1050px, since this value is obeyed by both headless and headed mode.

[WolfgangFellger]

@sebinsua We experienced the same issue. Passing --window-size seems to do the trick (at least in cypress 3.7) without the other shenanigans mentioned in this thread:

on('before:browser:launch', (browser = {}, args) => {
    if (browser.name === 'chrome') {
        args.push('--window-size=1920,1080')
        return args
    }
})

[memee]

You can pass a -screen option to xvfb-run specifying the resolution. See XVFB docs

Code where Cypress passes 1280x1024x8 option: https://github.com/cypress-io/cypress/blob/develop/packages/server/test/scripts/run.js#L55

−screen screennum WxHxD
This option creates screen screennum and sets its width, height, and depth to W, H, and D respectively. By default, only screen 0 exists and has the dimensions 1280x1024x8.

@jennifer-shehane I would rather not modify cypress code. Is there a recommended way to pass custom resolution to Xvfb with cypress? In plugins maybe. Besides, I'm using image: cypress/browsers:chrome69 in gitlab-ci and cypress-plugin-snapshot. I found the similar issue there: https://github.com/meinaart/cypress-plugin-snapshots/issues/71

[warpech]

I got lucky with resolution 1980x1080 in headless Chrome in Docker, without messing with xvfb.

I used the Docker image cypress/included version 3.8.3. My config:

1. In cypress.json, set:
   json "viewportWidth": 1920, "viewportHeight": 1080
2. In cypress/plugins/index.js, set:
   js on('before:browser:launch', (browser = {}, args) => { if (browser.name === 'chrome') { args.push('--window-size=1920,1080'); return args; } });
3. In the tests, set cy.viewport(1920, 1080);
4. My Docker-compose YML file that I use for running Cypress:
   yml version: '3.5' services: cypress: image: "cypress/included:3.8.3" environment: - CYPRESS_baseUrl=http://host.docker.internal:8080 - CYPRESS_cmd working_dir: /app volumes: - ../:/app command: --browser chrome --headless ${CYPRESS_cmd}

${CYPRESS_cmd}` is used to pass any additional command options, such as the name of the test file to run.

---

As mentioned in the beginning, this works with the resolution 1920x1080. At first, I tried 3840x2160 but cypress run froze at executing the first test. Later, I tried 2560x1440, but cypress run was still freezing at some tests 50% of the time. I settled on 1920x1080 and did not investigate further.

[RichieRunner]

I got lucky with resolution 1980x1080 in headless Chrome in Docker, without messing with xvfb.

I used the Docker image cypress/included version 3.8.3. My config:

1. In cypress.json, set:
   json "viewportWidth": 1920, "viewportHeight": 1080
2. In cypress/plugins/index.js, set:
   js on('before:browser:launch', (browser = {}, args) => { if (browser.name === 'chrome') { args.push('--window-size=1920,1080'); return args; } });
3. In the tests, set cy.viewport(1920, 1080);
4. My Docker-compose YML file that I use for running Cypress:
   yaml version: '3.5' services: cypress: image: "cypress/included:3.8.3" environment: - CYPRESS_baseUrl=http://host.docker.internal:8080 - CYPRESS_cmd working_dir: /app volumes: - ../:/app command: --browser chrome --headless ${CYPRESS_cmd}

${CYPRESS_cmd}` is used to pass any additional command options, such as the name of the test file to run.

> As mentioned in the beginning, this works with the resolution 1920x1080. At first, I tried 3840x2160 but cypress run froze at executing the first test. Later, I tried 2560x1440, but cypress run was still freezing at some tests 50% of the time. I settled on 1920x1080 and did not investigate further.

Still doesn't work for me. There are still differences with just running run and open locally. This is just silly.

[warpech]

Still doesn't work for me. There are still differences with just running run and open locally. This is just silly.

Yeah, for that reason I never create screenshots locally. I only use cypress gui locally for debugging without screenshots, or docker-compose to run Cypress in Docker with screenshots. That setup gives me the same screenshots in Windows, Mac, Linux and CI (GitHub Actions).

[RichieRunner]

Yeah, for that reason I never create screenshots locally. I only use cypress gui locally for debugging without screenshots, or docker-compose to run Cypress in Docker with screenshots. That setup gives me the same screenshots in Windows, Mac, Linux and CI (GitHub Actions).

oh, so do you mean create the baseline images FIRST through your CI build pipeline? Interesting. Then where/how would you find it to commit into into your source control?

[warpech]

oh, so do you mean create the baseline images FIRST through your CI build pipeline? Interesting. Then where/how would you find it to commit into into your source control?

No, I create the images on my developer machine using Chrome in Docker and commit them into the source control. The CI pipeline runs the tests and compares the screenshots in the source control with the current app using the plugin https://github.com/palmerhq/cypress-image-snapshot

[laurogripa]

Possibly related: electron#21379: Screen dimensions are off by one pixel in offscreen mode

[lunaboona]

Yeah, for that reason I never create screenshots locally. I only use cypress gui locally for debugging without screenshots, or docker-compose to run Cypress in Docker with screenshots. That setup gives me the same screenshots in Windows, Mac, Linux and CI (GitHub Actions).

oh, so do you mean create the baseline images FIRST through your CI build pipeline? Interesting. Then where/how would you find it to commit into into your source control?

Not necessarily take the screenshots in the CI pipeline, but rather use the same docker image in CI and locally to generate the base screenshots.

[iamjoncannon]

Wanted to update the @warpech solution above for headless Chrome. I spent some time on this yesterday.

Set up is Cypress 4.5 with cypress-image-snapshot in Docker (not using a cypress pre-built image).

I am running VRTs across viewports, and ran into this bug-- the tests were all coming out at 1280px and below, even with the viewport set at higher widths.

The following works at viewports above 1920--

cypress.json

{
    "viewportWidth": 2560, 
    "viewportHeight": 1080
}

cypress/plugins/index.js

on('before:browser:launch', (browser = {}, launchOptions) => {

    if (browser.name === 'chrome') {
      launchOptions.args.push('--disable-dev-shm-usage')
      launchOptions.args.push('--window-size=2560,1080')
      return launchOptions
    }
  })

When I tried @warpech solution initially the suite froze after taking the 1920 shot, but "disable-dev-shm-usage" ended up resolving the issue-- the suite loads at 2560 and reduces the screensize correctly for each it block. (Lower viewports are less memory intensive, which is why running at default 1280 works even without the shm flag)

@jennifer-shehane It might make sense to add this information here-- https://docs.cypress.io/api/plugins/browser-launch-api.html#Set-screen-size-when-running-headless-Chrome-- and/or here-- https://docs.cypress.io/guides/guides/continuous-integration.html#In-Docker -- and make it a bit more explicit that this is the solution to this issue.

The basic idea is-- headless Chrome in CI needs to load at the max viewport width the test will require, and from there it will resize properly during the run. Disabling shm is not only recommended but likely required for higher viewports.

[FDiskas]

Got it working by following the docs https://docs.cypress.io/api/plugins/browser-launch-api.html#Set-screen-size-when-running-headless

    if (browser.name === 'electron') {
      // fullPage screenshot size is 768x1024
      launchOptions.preferences.width = 768;
      launchOptions.preferences.height = 1024;
      launchOptions.preferences.frame = false;
      launchOptions.preferences.useContentSize = true;
    }