Yarn: [0.24.5] Issues with node-sass

[jiaweihli]

What is the current behavior?
node-sass, and other libraries with build artifacts, occasionally fail at runtime when installed with yarn. The workaround is to run npm rebuild node-sass after doing a yarn update.

If the current behavior is a bug, please provide the steps to reproduce.

Quoting a post on the same issue from 0.17.0:

How to reproduce consistently:

CircleCI 2.0 Docker program.

1. Create your project as is with a package.json. Create yarn lock file via yarn install/upgrade.
2. Add node-sass or gulp-sass into project (Currently testing with gulp-sass)
3. Ensure proper tokens are hooked into repository to allow webhooks for CircleCI
4. Ensure project is configured to use CircleCI Beta 2.0 (Apply on their website)
5. Add the following into .circleci/config.yml.
6. Check in code.

The first run, the docker container will be built, no cache will exist, yarn will run an install, and then perform a successful compilation of sass using gulp-sass. It will succeed.

Do another noop checkin. The following build will use the cache saved from the previous successful run, restore the cache, and then run a yarn install which will be very quick because cache has been restored. The next compilation command (the SASS/SCSS), will now consistently fail with:

yarn run css
yarn run v0.23.4
$ gulp css
fs.js:914
  return binding.readdir(pathModule._makeLong(path), options.encoding);
                 ^

Error: ENOENT: no such file or directory, scandir '/root/project/node_modules/node-sass/vendor'

./.circleci/config.yml

version: 2
jobs:
  build:
    working_directory: ~/project

    docker:
      - image: node:7.10

    steps:
      - checkout
      - restore_cache:
          key: deps-{{ .Branch }}-{{ checksum "yarn.lock" }}
      - run:
          command: yarn install --pure-lockfile --no-progress
      - save_cache:
          key: deps-{{ .Branch }}-{{ checksum "yarn.lock" }}
          paths:
            - node_modules
# ADD THIS COMMAND HERE THAT USES NODE-SASS / GULP-SASS
#      - run:
#          command: yarn run css  

Refs:

• sass/node-sass#1804

What is the expected behavior?
node-sass shouldn't need to be rebuilt via npm rebuild after an install/update. npm doesn't require a rebuild when installing it.

Please mention your node.js, yarn and operating system version.
node 6.10.2
yarn 0.24.5
mac 10.12.5

Closing notes
This was tackled in 0.24.0 via https://github.com/yarnpkg/yarn/pull/3224, but there looks to be outstanding issues.

This also affects other libraries - any library with artifacts - including karma-phantomjs-launcher.

[bestander]

linking https://github.com/yarnpkg/yarn/issues/3274

[bestander]

cc @xzyfer, who offered helping with making Yarn work with node-sass.

So, correct me if I am wrong, Yarn runs install scripts of node-sass package which generates node-sass/vendor folder.
Turns out, sometimes this folder gets deleted by Yarn?

Does anyone have a repro step?

[xzyfer]

I'm currently on holidays but will be back to normal activity next week.
IIRC the repro is to installal node-sass, rm node_modules, and run yarn
install again.

On 24 May 2017 5:47 pm, "Konstantin Raev" [e-mail] wrote:

cc @xzyfer https://github.com/xzyfer, who offered helping with making
Yarn work with node-sass.

So, correct me if I am wrong, Yarn runs install scripts of node-sass
package which generates node-sass/vendor folder.
Turns out, sometimes this folder gets deleted by Yarn?

Does anyone have a repro step?

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/yarnpkg/yarn/issues/3485#issuecomment-303765961, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AAjZWDvbnWfs-29ngeut01nQ8dML0DQVks5r9FEOgaJpZM4NkLLh
.

[acburdine]

I've got a repro that I've hit 3 or 4 times in a row with the same steps.

My project uses gulp-sass, which uses node-sass, so my project doesn't depend on node-sass explicitly, but I've ran into this issue with ember-cli-sass as well on another project so I don't know if it has anything to do with sub-dependencies.

Steps:

1. yarn install (with no node_modules folder)
2. yarn add <dep> (also happens when running yarn upgrade-interactive)
3. Run task that uses gulp-sass
4. See error above
5. rm -rf node_modules
6. yarn install
7. gulp-sass works again.

Have hit this same error every time I've done the yarn add or yarn upgrade-interactive in recent memory.

Let me know if there's any additional info I can provide such as gulp config, package.json, or anything else 😄

[bestander]

Thanks!

On 25 May 2017 at 00:45, Austin Burdine [e-mail] wrote:

I've got a repro that I've hit 3 or 4 times in a row with the same steps.

My project uses gulp-sass, which uses node-sass, so my project doesn't
depend on node-sass explicitly, but I've ran into this issue with
ember-cli-sass as well on another project so I don't know if it has
anything to do with sub-dependencies.

Steps:

1. yarn install (with no node_modules folder)
2. yarn add (also happens when running yarn upgrade-interactive)
3. Run task that uses gulp-sass
4. See error above
5. rm -rf node_modules
6. yarn install
7. gulp-sass works again.

Have hit this same error every time I've done the yarn add or yarn
upgrade-interactive in recent memory.

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/yarnpkg/yarn/issues/3485#issuecomment-303883478, or mute
the thread
https://github.com/notifications/unsubscribe-auth/ACBdWFaEmv2KuE6Mk0l3vTgHhsBqbheOks5r9MEygaJpZM4NkLLh
.

[ghost]

I've put the following repository together to reproduce the issue - https://github.com/destroyerofbuilds/yarn-sass

Developers at the company I work for actually run into the issue when using --force-lockfile, but I haven't been able to create a reproducer for that flag.

[blimmer]

@bestander I'm able to consistently reproduce this problem on the latest version of yarn (0.24.6) with these steps (also reported here):

git clone -b bug/show-issue-with-nodesass https://github.com/blimmer/ember-cli-example-app-for-github.git
cd ember-cli-example-app-for-github
yarn install --no-lockfile
DEBUG=ember-try* node_modules/.bin/ember try:one ember-lts-2.4 test --skip-cleanup

failing OSS build: https://travis-ci.org/blimmer/ember-cli-example-app-for-github/jobs/235289348

[blimmer]

Here's some more info after I did some investigation:

what this ember-try thing is doing is replacing the original package.json file with this diff:

diff --git a/package.json b/package.json
index 4fef7cb..63cf67b 100644
--- a/package.json
+++ b/package.json
@@ -2,9 +2,7 @@
   "name": "ember-try-test",
   "version": "0.0.0",
   "description": "The default blueprint for ember-cli addons.",
-  "keywords": [
-    "ember-addon"
-  ],
+  "keywords": ["ember-addon"],
   "license": "MIT",
   "author": "",
   "directories": {
@@ -38,7 +36,6 @@
     "ember-export-application-global": "^2.0.0",
     "ember-load-initializers": "^1.0.0",
     "ember-resolver": "^4.0.0",
-    "ember-source": "~2.13.0",
     "ember-welcome-page": "^3.0.0",
     "loader.js": "^4.2.3"
   },

From a totally clean repo, if I run:

> yarn install --no-lockfile
> ls node_modules/node-sass/vendor/darwin-x64-46
binding.node
# apply the package.json diff
> yarn install
> ls node_modules/node-sass/vendor/darwin-x64-46
gls: cannot access 'node_modules/node-sass/vendor/darwin-x64-46': No such file or directory

See how the bindings disappear there? The verbose output from the second yarn install that removes the bindings is here.

however, if I don't use --no-lockfile at first:

> yarn install
> ls node_modules/node-sass/vendor/darwin-x64-46
binding.node
# apply the package.json diff
> yarn install
> ls node_modules/node-sass/vendor/darwin-x64-46
binding.node

the file remains. verbose output from this is here.

Checking the version installed by both --no-lockfile and regular install both report the same version:

yarn list node-sass
yarn list v0.24.6
└─ [email protected]
✨  Done in 0.98s.

Not sure if this helps, but thought I'd report what I know!

[mikew]

I get the error when using --pure-lockfile, I haven't tried --no-lockfile, but without any args yarn install works.

[nschonni]

I wonder if yarn could use the files element in package.json to figure out that there is a folder it should keep https://github.com/sass/node-sass/blob/master/package.json#L45

[ghost]

I updated my comment above to reflect that the issue I was originally having was not with --frozen-lockfile, but solely with --pure-lockfile.

We've been running into problems while using the NodeJS buildpack, and that buildpack uses --pure-lockfile - https://github.com/heroku/heroku-buildpack-nodejs/blob/c883f50251537f0e4cccd95749c79dd044a4357f/lib/dependencies.sh#L67

[bestander]

Thanks, we are looking into the issue now.
A fix is coming next week

On Fri, 2 Jun 2017 at 20:15, Hutson Betts [e-mail] wrote:

I updated my comment above to reflect that the issue I was originally
having was not with --frozen-lockfile, but solely with --pure-lockfile.

We've been running into problems while using the NodeJS buildpack, and
that buildpack uses --pure-lockfile -
https://github.com/heroku/heroku-buildpack-nodejs/blob/c883f50251537f0e4cccd95749c79dd044a4357f/lib/dependencies.sh#L67

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/yarnpkg/yarn/issues/3485#issuecomment-305885115, or mute
the thread
https://github.com/notifications/unsubscribe-auth/ACBdWHQv495IGvwoZd3YbeK-ZVSPx89dks5sAF9OgaJpZM4NkLLh
.

[sth]

A related ticket with steps to reproduce the issue: #3506

[selfagency]

I don't know if this is the right place to put this, but since updating to the latest yarn, every time I run yarn in my project's folder, which uses nvm, I have to rebuild node-sass with npm because the bindings are to my system's version of node instead of the nvm installed version my project uses.

[jpommerening]

Ok guys, this problem bothered me for the last few days so I did some digging.

Here's what I found:

• The removal of the node-sass/vendordirectory is triggered in package-linker.js, here
• The possibleExtraneous files set (which the vendor dir ended up in) is populated by buildActionsForCopy and buildActionsForHardlink in util/fs.js (gloriously duplicated code btw 🙈)
• There is already some logic to exclude files from being deleted but I'm not exactly sure how the artifactFiles set (the code checks this for files _not_ to delete) is filled… maybe this would be the right place to implement what @nschonni suggested.

Also, I _assume_ there's something that checks every file whether exists in the yarn cache and puts files into the possibleExtraneous set. (The vendor directory is not present in the cache. I believe that's correct, because caching V8 bindings would break every time someone switches node versions with nvm use.)

If it's not an option to use the files entry in the package.json, maybe yarn could record files that were created by postInstall in the yarn.lock file? I'm not familiar enough with the inner workings of yarn to go any further, maybe someone can take over? Anyway, hope this helps!

[bestander]

Yarn should record the files int .yarn-integrity file, something is getting lost over there I suppose.

[bestander]

As @destroyerofbuilds described in https://github.com/destroyerofbuilds/yarn-sass, the issue was that --pure-lockfile resulted in no .yarn-integrity file saving.
Fix is coming

[bestander]

Merging this issue into #3506

[selfagency]

I wasn't using --pure-lockfile to install anything when Yarn broke my node-sass bindings. Should I open a separate issue?

[bestander]

@selfagency this is another issue.
The reason is that we don't ship node-gyp with Yarn and rely on the global one.
Probably something is wrong with the paths

[ghost]

@bestander has merged in #3704 to resolve this issue.

Using the nightly 0.27 build (0.27.0-20170623.1327), which contains the fix, I no longer get the missing file error (Error: ENOENT: no such file or directory).

Thank you @bestander

[jiaweihli]

The original issue I filed wasn't related to #3506 - I get this issue despite using none of --pure-lockfile, --no-lockfile, or --frozen-lockfile.

Can you de-merge this from that and re-open this?

[bestander]

@jiaweihli to avoid confusion would you please open an issue with repro steps?

[ghost]

with latest

node -v
    v8.1.2
yarn --version
    0.27.0

adding latest node-gyp

yarn global add node-gyp
yarn global list
    yarn global v0.27.0
    warning package.json: No license field
    warning No license field
    info "[email protected]" has binaries:
       - node-gyp

initial node-sass install is OK

yarn add --dev node-sass@^4.5
yarn list --depth 0 node-sass
    yarn list v0.27.0
    └─ [email protected]

as is upgrade, tho it takes quite awhile 'Rebuilding all packages',

yarn upgrade node-sass@^4.5
    [4/4] Rebuilding all packages...
    success Saved lockfile.
    success Saved 0 new dependencies.
    Done in 119.06s.

I haven't yet been able to cause another similar node-sass related fail ... so far (fingers crossed)

[jontelm]

"npm rebuild node-sass" was a way around for now.

[spullara]

I also got around it with "yarn upgrade node-sass" which built it and then didn't later delete it.

[mzabuawala]

@jontelm Thanks a lot, after messing around for 3 hours your suggestion worked for me.
Solution for me: "npm rebuild node-sass"