yarn install silently deletes installed files

[ckuetbach]

Bug

yarn install silently deletes installed files.

What is the current behavior?
The project is not complete after yarn install.

steps to reproduce

• yarn install
• yarn check
  package.json.txt

If I remove gulp-typings there will also be all files installed. I can see, that all files have been placed during install and deleted during installation of typings-global.

If I install these packages with npm install everything works as expected.

What is the expected behavior?
The files within the browserify modules are still present after installing.

Please mention your node.js, yarn and operating system version.
yarn: 1.3.2
npm: 5.5.1
node: v8.9.1

[ckuetbach]

I use Windows 10 (Version 1709) (current version)

[rally25rs]

I seem to be unable to reproduce this using your package.json file:

$ cat package.json
{
  "name": "test",
  "version": "0.0.1",
  "description": "Bugreport",
  "author": "[email protected]",
  "license": "na",
  "repository": {
    "url": "na"
  },
  "devDependencies": {
    "browserify": "14.5.0",
    "gulp-typings": "2.0.4"
  },
  "dependencies": {

  }
}

$ yarn install
yarn install v1.3.2
warning package.json: License should be a valid SPDX license expression
info No lockfile found.
warning [email protected]: License should be a valid SPDX license expression
[1/4] 🔍  Resolving packages...
[2/4] 🚚  Fetching packages...
[3/4] 🔗  Linking dependencies...
[4/4] 📃  Building fresh packages...
success Saved lockfile.
✨  Done in 8.53s.

$ yarn check
yarn check v1.3.2
warning package.json: License should be a valid SPDX license expression
warning [email protected]: License should be a valid SPDX license expression
success Folder in sync.
✨  Done in 0.57s.

$ ls -l node_modules
total 0
drwxr-xr-x   3 jvalore  891112136  102 Nov 10 21:45 @types
drwxr-xr-x  11 jvalore  891112136  374 Nov 10 21:45 JSONStream
drwxr-xr-x   6 jvalore  891112136  204 Nov 10 21:45 abbrev
drwxr-xr-x  11 jvalore  891112136  374 Nov 10 21:45 acorn
drwxr-xr-x  11 jvalore  891112136  374 Nov 10 21:45 agent-base
drwxr-xr-x  17 jvalore  891112136  578 Nov 10 21:45 any-promise
drwxr-xr-x   8 jvalore  891112136  272 Nov 10 21:45 array-filter
drwxr-xr-x   9 jvalore  891112136  306 Nov 10 21:45 array-map
... more output omitted for brevity ...
drwxr-xr-x   7 jvalore  891112136  238 Nov 10 21:45 zip-object

$ ls node_modules/browserify
LICENSE            changelog.markdown example            lib                package.json       test
bin                code-of-conduct.md index.js           node_modules       readme.markdown

Do you have a .yarnclean file in your project anywhere? That could potentially cause files to be removed.

[ckuetbach]

A Windows only Bug?
err.txt

I was able to reproduce this at a second maschine with a fresh install of node and yarn.
```
Volume in Laufwerk C: hat keine Bezeichnung.
Volumeseriennummer: 761D-396A

Verzeichnis von C:\Users\christian\Downloads\yarnbug\node_modules

12.11.2017 20:13

.
12.11.2017 20:13 ..
12.11.2017 20:13 .bin
12.11.2017 20:13 38.937 .yarn-integrity
12.11.2017 20:13 @types
12.11.2017 20:13 astw
12.11.2017 20:13 browser-pack
12.11.2017 20:13 browserify
12.11.2017 20:13 configstore
12.11.2017 20:13 create-hash
12.11.2017 20:13 create-hmac
12.11.2017 20:13 deps-sort
12.11.2017 20:13 detective
12.11.2017 20:13 diffie-hellman
12.11.2017 20:13 insert-module-globals
12.11.2017 20:13 invariant
12.11.2017 20:13 module-deps
12.11.2017 20:13 pbkdf2
12.11.2017 20:13 shasum
12.11.2017 20:13 smartshell
12.11.2017 20:13 syntax-error
12.11.2017 20:13 touch
12.11.2017 20:13 typings-core
12.11.2017 20:13 typings-global
1 Datei(en), 38.937 Bytes
23 Verzeichnis(se), 29.391.728.640 Bytes frei
Volume in Laufwerk C: hat keine Bezeichnung.
Volumeseriennummer: 761D-396A

Verzeichnis von C:\Users\christian\Downloads\yarnbug\node_modules\browserify

12.11.2017 20:13

.
12.11.2017 20:13 ..
12.11.2017 20:13 node_modules
0 Datei(en), 0 Bytes
3 Verzeichnis(se), 29.390.180.352 Bytes frei

`

[rally25rs]

That's really weird. I tried it on my SurfacePro3 / Win 10 / Powershell and it still seems OK for me:

C:\Projects\yarn-test> more .\package.json
{
  "name": "test",
  "version": "0.0.1",
  "description": "Bugreport",
  "author": "",
  "license": "na",
  "repository": {
    "url": "na"
  },
  "devDependencies": {
    "browserify": "14.5.0",
    "gulp-typings": "2.0.4"
  },
  "dependencies": {

  }
}

C:\Projects\yarn-test> yarn
yarn install v1.3.2
warning package.json: License should be a valid SPDX license expression
info No lockfile found.
warning [email protected]: License should be a valid SPDX license expression
[1/4] Resolving packages...
[2/4] Fetching packages...
[3/4] Linking dependencies...
[4/4] Building fresh packages...
success Saved lockfile.
Done in 19.70s.

C:\Projects\yarn-test> ls .\node_modules\browserify


    Directory: C:\Projects\yarn-test\node_modules\browserify


Mode                LastWriteTime         Length Name
----                -------------         ------ ----
d-----       11/12/2017   8:45 PM                bin
d-----       11/12/2017   8:45 PM                example
d-----       11/12/2017   8:45 PM                lib
d-----       11/12/2017   8:45 PM                node_modules
d-----       11/12/2017   8:45 PM                test
-a----        10/9/2017   5:51 PM             76 .travis.yml
-a----       10/20/2017   5:00 AM          39281 changelog.markdown
-a----        10/9/2017   5:51 PM           3256 code-of-conduct.md
-a----       10/20/2017   4:47 AM          24259 index.js
-a----        10/9/2017   5:51 PM           2802 LICENSE
-a----       10/20/2017   5:01 AM           2283 package.json
-a----        10/9/2017   5:51 PM          26255 readme.markdown

Unfortunately I'm not really sure how else to try to reproduce it. Would it be possible to make a full sample project in a git repository that I can use to try to reproduce it?

You might try running yarn install --verbose and see if there are any clues in the output.

[ckuetbach]

I reproduced it with an empty folder, with nothing more, than the package.json in it.

As long as I can see, everything is OK with the installation. But during [4/4] building fresh packages the files will be deleted.

This is th only interesting line, I think: verbose 15.329 @types/node not yet installed
out.txt

My other maschine was not connected to my companies NPM-package proxy. But I will create a verbose-log with that maschine or a new VM soon.

[rally25rs]

I think that "building fresh packages" step runs the build scripts for the packages. Try running yarn install --ignore-scripts and see if the files still get deleted.

[ckuetbach]

With this flag all files are present.

This will lead to the question, why you was´t able to reproduce this bug, because all referenced modules had a hard version. So your scripts should have been the same as mine.

One of my mashine had nothing preinstalled, but node and yarn in the most recent (released) version.

[rally25rs]

So I tried to reproduce this again today, and suddenly I am seeing the same thing as you; files being deleted. 😕

Trying to dig into this, my guess at this point is that gulp-typings has this postinstall script: https://gitlab.com/pushrocks/typings-global/blob/master/dist/install.js

That does a cd .. && npm install --no-save @types/node@">=${nodeMajorVersion}.0.0 <${nodeNextVersion}.0.0"
which produces the output:

npm WARN gentlyRm not removing /Users/jvalore/Projects/yarn-test/node_modules/.bin/semver as it wasn't installed by /Users/jvalore/Projects/yarn-test/node_modules/semver
npm WARN [email protected] license should be a valid SPDX license expression

+ @types/[email protected]
added 1 package and removed 234 packages in 2.215s

I'm starting to think that NPM deletes a bunch of stuff ("removed 234 packages") but I'm not entirely sure if that is the case, since something else seems to use "gentlyRm" to delete things as well.

I'll try to keep digging in as I have time...

[rally25rs]

Alright @ckuetbach you've stumbled upon a bigger issue here...

~/Projects/yarn-test : yarn
yarn install v1.3.2
warning package.json: License should be a valid SPDX license expression
warning [email protected]: License should be a valid SPDX license expression
[1/4] 🔍  Resolving packages...
[2/4] 🚚  Fetching packages...
[3/4] 🔗  Linking dependencies...
[4/4] 📃  Building fresh packages...
✨  Done in 0.87s.

~/Projects/yarn-test : ls node_modules/browserify
LICENSE            changelog.markdown example            lib                package.json       test
bin                code-of-conduct.md index.js           node_modules       readme.markdown

~/Projects/yarn-test : npm install left-pad
npm WARN [email protected] license should be a valid SPDX license expression

+ [email protected]
added 1 package, removed 133 packages and updated 1 package in 1.359s

~/Projects/yarn-test : ls node_modules/browserify
node_modules

It looks like if you NPM install anything in an existing Yarn-installed directory, then NPM deletes the content of all the non-npm installed packages.

In your case, as I mentioned above a postinstall script run an npm install command.

This... seems bad 😞 Although I don't think there is anything Yarn can do about it, since it's during an NPM install that the deletions happen.

[rally25rs]

Going to mark this closed as an NPM issue:

• https://github.com/npm/npm/issues/16794
• https://github.com/npm/npm/issues/17379
• https://github.com/npm/npm/issues/16853

And I don't think there is any way this can be fixed by Yarn.

[ckuetbach]

Thanks a lot for digging into this. Seems not to be a yarn bug, but there is also nothing a package consumer can do against this.

Somtimes I think web-dev was easier, as there were only IE bugs to handle.

[lordmauve]

There may be something you can do. In the shell you use for running the postinstall script (and only in that shell), provide an 'npm' executable that is either an alias for /bin/false, or executes yarn in some npm-compatibility mode.

Hacky solution, of course, but also the kind of thing package managers sometimes have to do to deal with the real world.

[rachaelnelson]

It's been a while since this thread was active, but wanted to confirm that I am having this problem with npm v 6.12.0 and yarn 0.15.1. So frustrating. Could not figure out where the heck my git tracking disappeared after a yarn install, HOWEVER, I am not able to reproduce this all the time. Very strange.