Cypress: Proposal: download Cypress to the common cached folder

[bahmutov]

Feature

Instead of downloading and unzipping Cypress into ./node_modules/cypress/dist/ download it into a global folder for better caching. This will allow

• better caching of dependencies on CI
• reusing same version of Cypress if already downloaded once for some project

To determine output location use https://github.com/LinusU/node-cachedir

var cachedir = require('cachedir')
var path = cachedir('cypress')
// `path` now contains the path under which you should store cached files

• cypress run, open, verify

  • [ ] ~take arg --binary-folder (looks like: /foo/bar/Cypress.app (macOS) or /foo/bar/Cypress (linux, win), to override default path to cypress binary folder (default is (cypressCache)/<cli version>/Cypress.app)~
  • [x] can override folder of binary with env var CYPRESS_BINARY_FOLDER=/path/to/Cypress.app
  • [x] warn user if set

• cypress install

  • [x] Checks (cypressCache)/<cli version> for already installed binary folder (Cypress.app on macOS, Cypress on linux/win), and if binary version matches (installVersion), do nothing.
  • [x] (installVersion) defaults to cli version, is overridden by env var CYPRESS_BINARY_VERSION during install
  • can be either semvar: 1.2.3, url: www.cypress.io/download/1.2.3, or zip: /path/to/cypress.zip
  • [x] if cypress install --force, force reinstall to (cypressCache)/<cli version>
  • [x] (cypressCache) defaults to:
  • ~/Library/Caches/Cypress (macOS)
  • ~/.cache/Cypress (linux)
  • /AppData/Local/Cypress/Cache (win)
  • [x] (cypressCache) can be overridden with env var CYPRESS_CACHE_DIRECTORY
  • [x] before install, ensure write access to (cypressCache)

• [x] custom error message for failed access to (cypressCache)

• [ ] docs
  
  
  
  • [x] Installing Cypress
  
  
  • [ ] ~command line usage~
  
  
  • [ ] update info about global install
  
  

• [x] manual testing all use cases

  • [x] cypress install
  • CYPRESS_BINARY_VERSION
    
    
    
    • [x] url
    
    
    • [x] skip if has any cache
    
    
    • [x] semvar
    
    
    • [x] zip
    
    
    • [x] skip if has any cache
    
    
    • [x] show progress for unzipping
    
    
    • [x] --force
    
    
  • [x] --force
  • [x] ~--binary-folder~
  • [x] ~cypress run~
  • [x] ~cypress open~
  • [x] ~cypress verify~
  • [x] ~invalid~
  • [x] CYPRESS_CACHE_DIRECTORY
  • valid
  • invalid, error message

• [x] add local code coverage with nyc

  • [ ] send to coveralls in CI
    

    Stretch goal

• [ ] can we cache ffmpeg separately to avoid bundling the same version over and over?

[ntwb]

Either this is a duplicate of #1233 or vice-versa, not overly concerned which issue is closed but they are for the most part duplicates

[zkat]

psst: If you use make-fetch-happen you'll be able to tie the cypress download cache to npm's own cache (by pointing it at process.env.npm_config_cache in the run script).

[EirikBirkeland]

The Cypress.io download always doubles the time my yarn install takes, on a 25 mbit connection. There are ways to get around, but would greatly appreciate caching.

[bahmutov]

Can you cache node_modules ?

Sent from my iPhone

On Mar 20, 2018, at 11:22, Eirik Birkeland [e-mail] wrote:

The Cypress.io download always doubles the time my yarn install takes, on a 25 mbit connection. There are ways to get around, but would greatly appreciate caching.

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub, or mute the thread.

[ncjones]

To work around this issue we build a docker image with our test suite's dependencies pre-installed.

[KellyR-STCU]

Would love to see this put into a global location. My team is really excited about Cypress, but we've got to optimize the build time. I tried out npm ci and that actually shaved 2 minutes off the install process. Preventing this download from needing to occur repeatedly would shave about another 40-45 seconds off.

[EirikBirkeland]

@bahmutov I can cache node_modules on CI server, but for local builds we often change our dependencies because we are jumping between branches.

[garytryan]

Also having this problem with slow install times for local development using Yarn.

[Bkucera]

currently in progress in PR #1582

(my hard drive needed the space) -->

[Bkucera]

This also enables us to ship our docker images with Cypress pre-cached, which will help simplify CI setup!
Opened issue in docker images

[brian-mann]

Yeah it's true but we'll need to update our build process to include it - but we should it's a good idea.

Open a separate issue for that and we can work on that while you finish this PR.

[bahmutov]

we need to think about caching Cypress in the docker container - because it takes up a lot of space. If we do second split (Electron vs our code), then YES, @Bkucera that would be super good idea to include multiple versions pre-cached because they would not add much weight

[brian-mann]

@bahmutov good point... that's another "+" in the corner of splitting out those two things vs caching the entire app bundle.

[brian-mann]

Released in 3.0.0.

[maxime1992]

According to Gleb's comment, this issue should include:

better caching of dependencies on CI
reusing same version of Cypress if already downloaded once for some project

I just tried to:

• upgrade cypress within my project to 3.0
• make sure that ~/.cache/Cypress/3.0.0 exists
• rm -rf node_modules
• yarn again

Cypress has been downloaded again instead of using the local version, available in cypress global cache folder. Am I missing something? Is that not the expected behaviour and maybe I should open a new issue?

That's a great feature, looking forward to it, congrats :)

[Bkucera]

@maxime1992 yep, that was a bug, fixed in 3.0.1 !

[jeserkin]

If one installs it globally on machine and versions match (with project), then installing with npm shouldn't pull it again and install on project level, correct?

[Bkucera]

@jeserkin correct

[jeserkin]

@Bkucera Is it possible, that installing dependencies with npm ci, somehow mitigates existence check and still downloads cypress based on project level description?
At the moment I have npm install -g [email protected] pre-installed on docker image. Later I run npm ci in container, that is based on given image and it does following:

HH:mm:ss  > [email protected] postinstall /build/xtss/node_modules/cypress
HH:mm:ss  > node index.js --exec install
HH:mm:ss  
HH:mm:ss  Installing Cypress (version: 3.3.2)
HH:mm:ss  
HH:mm:ss  [?25l[12:31:14]  Downloading Cypress     [started]
HH:mm:ss  [12:31:15]  Downloading Cypress      0% 0s [title changed]
HH:mm:ss  [12:31:15]  Downloading Cypress      3% 3s [title changed]
HH:mm:ss  [12:31:15]  Downloading Cypress      8% 2s [title changed]
HH:mm:ss  [12:31:15]  Downloading Cypress      12% 2s [title changed]
HH:mm:ss  [12:31:15]  Downloading Cypress      18% 2s [title changed]
HH:mm:ss  [12:31:15]  Downloading Cypress      24% 2s [title changed]
HH:mm:ss  [12:31:15]  Downloading Cypress      29% 1s [title changed]
HH:mm:ss  [12:31:16]  Downloading Cypress      34% 1s [title changed]
HH:mm:ss  [12:31:16]  Downloading Cypress      41% 1s [title changed]
HH:mm:ss  [12:31:16]  Downloading Cypress      47% 1s [title changed]
HH:mm:ss  [12:31:16]  Downloading Cypress      52% 1s [title changed]
HH:mm:ss  [12:31:16]  Downloading Cypress      60% 1s [title changed]
HH:mm:ss  [12:31:16]  Downloading Cypress      68% 1s [title changed]
HH:mm:ss  [12:31:16]  Downloading Cypress      74% 0s [title changed]
HH:mm:ss  [12:31:16]  Downloading Cypress      81% 0s [title changed]
HH:mm:ss  [12:31:16]  Downloading Cypress      89% 0s [title changed]
HH:mm:ss  [12:31:16]  Downloading Cypress      96% 0s [title changed]
HH:mm:ss  [12:31:16]  Downloaded Cypress      [title changed]
HH:mm:ss  [12:31:16]  Downloaded Cypress      [completed]
HH:mm:ss  [12:31:16]  Unzipping Cypress       [started]